DEV Community: Femi-ige Muyiwa

Run Postgres For Free: Top 3 Options

Femi-ige Muyiwa — Thu, 27 Mar 2025 12:29:29 +0000

“When life gives you lemons, TAKE THEM; free stuff is cool.” ~ Teo Deleanu.

And when life gives you PostgreSQL for free, you take that too. Why? Because taking free stuff isn’t just cool; it’s the smartest thing to do. Whether you’re a developer building a side project or a startup testing the waters, running PostgreSQL without breaking the bank is a win-win.

This article will show you the three top options for running PostgreSQL databases for free, from exploring some modern cloud hosting options to setting them up for development.

By the end of this article, you’ll know:

How do these platforms compare?
Which free-tier hosting platforms are the best (and what are their limitations)?

Why PostgreSQL?

PostgreSQL is a powerful open-source database that is constantly improved by a large community of developers. It supports familiar SQL queries, making it easy to use while offering the flexibility and reliability needed for modern full-stack apps. With over 35 years of open-source excellence, it is one of the greatest relational databases of all time, trusted by developers worldwide. It is a truly open-source project with no hidden licensing fees, making it a cost-effective choice for any project.

Also, PostgreSQL is known for its rock-solid data integrity as it follows the Atomicity, Consistency, Isolation, and Durability principles (ACID). In doing so, PostgreSQL never loses, corrupts, or allows inconsistent data, which is critical for applications like banking and healthcare.

One of PostgreSQL’s standout features is its concurrency handling through its Multi-Version Concurrency Control (MVCC). This feature allows multiple transactions to read the same data without blocking writes. Here is an SQL query that represents two database transactions running in separate sessions:

-- Transaction 1 (Session A)
BEGIN;
SELECT * FROM orders WHERE id = 1; -- Reads the current state
UPDATE orders SET status = 'shipped' WHERE id = 1;
COMMIT;

-- Transaction 2 (Session B)
BEGIN;
SELECT * FROM orders WHERE id = 1; -- Reads the old state before the update in Session A is committed
COMMIT;

With this technique, Session B sees a consistent snapshot of the data without being blocked by Session A’s update, making PostgreSQL a reliable choice for highly concurrent apps such as e-commerce, finance, and collaborative platforms.

Now, the moment you’ve been waiting for: here are the top three free-tier hosting solutions for the Postgres database. You’ll get to see why developers love them, as well as a quick start guide and some of their limitations.

Managed Postgres Comparison Table

Here is a like-for-like comparison table summarizing the features of some of the best free-tier Postgres hosting platforms.

Feature	Neon	Aiven	Render
Ease of Use	Easiest for beginners – Instant provisioning, database branching, and serverless architecture.	Moderate – Supports multiple open-source technologies but limited to one service on the free plan.	Moderate – Unified platform but limited to one free database instance.
Free Tier Storage Disk Space	0.5 GB	5 GB	1 GB
Compute Hours	191.9h/month	Not specified	Not specified
Postgres Versions	17, 16, 15, 14	17, 16, 15, 14, 13	16, 15, 14, 13
Database Branching	Yes (Full data branching and Schema-only branching BETA)	No	No
Cloud Providers	Multiple (AWS and Azure)	Single (DigitalOcean)	Single (AWS)
Limitations	Autoscaling can burn through compute hours.	Max 20 connections, no connection pooling.	30-day limit, no backups, single instance only.
Best For	large enterprise workloads as well as small projects, MVPs, and scalable applications.	Testing and small projects.	Small development and testing.

Neon

Neon provides a cloud-based, scalable implementation of PostgreSQL that can run on your physical hardware or through their managed services. Its architecture is separated into “compute” (for Postgres) and “storage” (a multi-tenant key-value store for Postgres pages), allowing Neon to gain true serverless capabilities.

Separating these architectures allows Neon to autoscale its compute resources depending on the activity period while maintaining data separately in a multi-tenant storage system. This architecture allows users to set up resources as soon as requested (instant provisioning), making it effective for managing costs since you only pay for the compute resources you use. This approach makes Neon reliable and cost-efficient compared to traditional database architectures, where storage and computing are tightly coupled.

Why Developers Love Neon
Neon offers a generous free tier option, enabling developers to build small projects and MVPs for a good period of time. Here are some of the benefits of the Neon PostgreSQL free tier:

You can have up to 10 projects, with each project able to own 10 branches.
Supports connection pooling through PgBouncer, enabling up to 10,000 concurrent connections.
Provides automatic backups through its "Point-in-Time Restore" (PITR) feature.
Database branching: This option allows you to create different environments (Dev, Staging, and Production) for your Postgres. You can also create some backup, as a new branch always contains data from the main branch.
The free tier offers a 0.5 GB storage limit, 191.9 compute hours/month, and 5GB of data transfer.
It supports Postgres versions 17, 16, 15, and 14.
Supports multiple cloud service providers and has access to all regions available to these providers.

Quick Start Guide
You can sign up to Neon using an email and password or with other third-party providers like Google, GitHub, or Microsoft. Then, create a new project using any configuration you prefer.

Limitation
Though autoscaling can improve performance during high-activity periods, it can easily burn through the compute hours during the free trial.

Aiven

Unlike Neon, Aiven provides many open-source managed data infrastructures, such as PostgreSQL Apache Cassandra, Apache Kafka, Apache Kafka Connect, Apache Kafka MirrorMaker 2, Elasticsearch, Grafana, InfluxDB, M3, M3 Aggregator, MySQL, and Redis. It’s a jack of all trades regarding its support for open-source technology.

Why Developers Love Aiven
Aiven supports deployment across various cloud providers, allowing users to choose the best infrastructure for their needs. Aiven offers access to its service using the free plan (PostgreSQL, MySQL, ValKey) or a 30-day free trial. When using the free plan, you can only access one service, but Aiven offers a 30-day free trial that can allow you to access all the services.

Aiven for PostgreSQL is a fully managed and hosted relational database service, and it offers features that developers can use indefinitely, such as:

It supports Postgres versions 17, 16, 15, 14, and 13.
The free tier consists of a single node, 1 CPU per virtual machine, 1 GB of RAM, and 5 GB of disk storage.
Management through the Aiven Console, CLI, API, Terraform Provider, or Kubernetes Operator.
Monitoring for metrics and logs.
Supports backups for disaster recovery.

Quick Start Guide
You can sign up to Aiven using an email or with any of its third-party authentication provider options. Afterward, you must log in to the console and follow the onboarding process to complete your registration.

Once you finish that, you are greeted with a page listing services you can create. Select the PostgreSQL service and set up your service using your desired configuration, then click ‘Create free service’ to deploy.

Limitations
Here are some of the limits to consider when using Aiven Postgres free tier:

It only supports a maximum of 20 connections, and connection pooling is not available to help manage multiple client connections.
Limited support: Aiven doesn’t provide access to dedicated support teams and, instead, relies solely on the Aiven Community Forum for user assistance.
The free plan is not covered by the Aiven Service Level Agreement.
A single cloud provider (DigitalOcean), limited service region, and service plan for the free plan.

Render

Render is a more traditional hosting platform that can build and deploy your applications, websites, and databases. As a unified platform, Render is an awesome choice for both seasoned developers and beginners, as you do not have to worry about using multiple platforms to host your application and your database(s). It can run various projects, from static sites and web applications to background workers, APIs, cron jobs, and managed databases.

Why Developers Love Render
Apart from Render’s user-friendly interface, it offers a free PostgreSQL hosting plan for development and testing, hobby projects, and MVPs. Here is an overview of some of its features and benefits:

Fixed Resource Allocation: The free PostgreSQL plan provides 256 MB RAM, 1 GB storage capacity, and 0.1 CPU, which can be enough for small-scale development.
It supports Postgres versions 16, 15, 14, and 13.
The free PostgreSQL plan has all regions available to other paid plans.
It allows you to integrate your PostgreSQL database with the other services available to Render. So, if you have an existing project hosted on Render and want to host a database for that project, Render gives you the option to do that during the database setup.

Quick Start Guide
Sign up to Render using an email or any third-party authentication provider option. Afterward, you will be greeted with various service types that Render offers. Select New PostgreSQL and set up the service with your desired configurations.

Limitations
Here are some of the limitations to consider when choosing the free tier option for Render:

The Free PostgreSQL database has a 30-day limit and will expire after that deadline. It has a 14-day grace period after the deadline, which will cause Render to delete the database if you do not update the plan.
It has a single instance limit for the PostgreSQL database. You can have only one active Free PostgreSQL database for any given workspace.
It doesn't support database backups.
Downtime or maintenance would cause the database to be temporarily unavailable.

Final Thoughts

Neon, Aiven, and Render are outstanding, cost-effective database options with great value. Each has standout features that, depending on your needs, can give you a great developer experience:

Neon stands out with its serverless architecture and developer-friendly features, such as database branching and generous compute hours. Its instant provisioning and transparent resource allocation make it perfect for large enterprises, rapid prototyping, and early-stage startups.
Aiven delivers reliability with its fixed 5GB storage and support for multiple Postgres versions. While the free tier is limited to one service, its monitoring tools and backup support make it ideal for learning and small production workloads.
Render shines with its unified platform approach, letting you host your applications and databases in one place. Though time-limited, its straightforward setup and integration capabilities make it an excellent choice for full-stack applications.

So, what are you waiting for? Pick the best platform for your project and start building today without spending a dime!

Resources

If you're considering using PostgreSQL the traditional way, here are guides to install it on macOS, Linux, or Windows:

5 Must-Watch Tutorials to Build Your SaaS App in 2025

Femi-ige Muyiwa — Thu, 27 Feb 2025 13:29:17 +0000

Today, building a Software as a Service (SaaS) application is easier than ever, at least on paper. With no-code tools like Webflow and serverless platforms handling the heavy lifting, you can spin up a functional product in record time. But getting from an idea to a successful launch? That’s where things get tricky.

It’s more than writing code. You need the right tools, a solid market approach, and a willingness to keep learning. The good news? If you know where to look, there’s a wealth of knowledge out there.

To simplify things, I’ve rounded up five must-watch video tutorials to guide you through the process. Whether you’re a seasoned developer or just testing the waters, these videos will help you build, launch, and grow a SaaS app that makes an impact.

1. Building a Finance Manager SaaS Platform

The finance industry is a large sector with many market opportunities. Whether you are an entrepreneur launching a fintech startup or an individual hoping to solve a market problem, building a finance app can provide people with better tools to track expenses, budget effectively, and make informed financial decisions. By creating a finance app, you empower users to take control of their money through features like automation and statistics, giving them a better idea of how their money comes in and goes out.

In this tutorial, you’ll learn how to build a finance manager app that tracks daily transactions, income, and expenses while visualizing them with clear graphs. You’ll also see how to add a point-in-time feature so users can analyze their data for specific date ranges. There’s a subscription-based element that unlocks premium features like connecting bank accounts and importing transactions, giving users more flexibility with their finances.

You’ll start by setting up Next.js for the project and then integrating route and API management using Hono. Shadcn UI makes UI prototyping faster, so you won’t have to build every component from scratch. The video also covers Clerk authentication for protected routes and demonstrates how Neon PostgreSQL fits in as a serverless solution for SQL databases.

You’ll use Drizzle ORM to manage TypeScript-to-Postgres mapping, so you will never have to juggle low-level SQL queries. Drizzle ORM helps maintain a safer interaction between your database and the Typescript project while enabling efficient schema management.

You won’t see how to connect bank accounts in this video, but you do get a neat way to import transactions. You can customize your table headers with predefined names, letting you decide which columns to keep or skip.

2. Build a SaaS App With Django, Stripe, Neon PostgreSQL, TailwindCSS, GitHub Actions

Knowing what tools, when, and how to use them remains invaluable in your programming journey. It is a huge part of being an innovator or pioneer and the foundation of being a great programmer.

This excellent tutorial will teach you how to build a foundational SaaS application using Django, covering authentication, data modeling, and payment integrations. You’ll understand why Python is a go-to language with its rich library ecosystem that streamlines common functionalities. There’s also a focus on setting up a scheduled production worker to sync user subscriptions at intervals.

You’ll start by configuring a Python project with Django to manage views, routes, and static files. The video also shows how to style the front end using Flowbite and TailwindCSS. The tutorial then walks you through preparing Django for a production environment and deploying it to Railway’s cloud platform via Docker containers.

The video explores subscription and payment integration in Stripe, showing how to handle recurring payments and manage user access based on their subscription status. Next, you’ll handle user authentication with Django Allauth by creating social authentication using GitHub authentication and linking user data with Stripe on registration.

This tutorial also highlights using PostgreSQL as the database option in the Django project, specifically focusing on Neon PostgreSQL. Neon is a managed form of PostgreSQL that eases development processes through its serverless architecture, allowing developers to focus on building rather than infrastructure management. The video tutorial emphasizes Neon's quick database setup and serverless approach, which means it can respond to the growing demand for your SaaS product, and the server only runs when you need it to run.

This is a beginner-friendly guide that helps you anticipate common pitfalls. You’ll learn to implement Stripe with Django in a way that gracefully handles subscription logic, user accounts, and data syncing, giving you a solid foundation to build and scale your SaaS product.

3. How To Build A $1,000,000 SaaS In 7 Hours

The saying “There is nothing new under the sun” might be true in some cases, but in others, innovation comes not from creating something entirely new but from improving existing ideas by addressing their weaknesses. Building a SaaS product doesn’t mean you must be original or a pioneer in the field of interest. In a competitive market, one or two key differences in its abilities could differentiate a good product from a great product. If you want to explore that, then this video is for you.

This video provides more insights into building SaaS solutions on already existing ones while improving on their weaknesses. It shows how to use the latest Next.js features to build a clone of ParityDeals, an application that offers discounts based on the user's country. The finished clone improves upon ParityDeals banner customization by introducing custom font sizes and manual CSS prefix configurations. There’s a permission-based system that differentiates users' access based on their subscriptions.

You’ll set up Next.js with Shadcn UI and TailwindCSS for styling and then configure Clerk for authentication. The video covers Neon Postgres database branching for development and uses Neon to store product data, customizations, and subscriptions. Drizzle ORM handles database setup, including schema, migrations, and credentials.

One of the best parts of this tutorial is its use of caching. It implements three levels, global, user-specific, and ID-specific, to optimize data storage and retrieval based on relevance.

Additionally, it revalidates cached data and updates it correctly when changes occur, such as product edits or deletions. This keeps information up to date without delays, making the tutorial a great resource for enhancing existing product features.

4. SaaS Automation Builder

Automation eliminates many cognitive or physical limitations humans face, making it useful for time-sensitive or repetitive tasks. This makes it crucial in production as it improves productivity while potentially saving on labor. This tutorial demonstrates how to build an automation platform called Fuzzie. The finished SaaS app allows users to create and edit workflows by setting a trigger (Google Drive) with different actions. Each workflow has a draggable canvas with a minimap and controls for centering and zooming in or out of the canvas page. Users can save their workflow to a database or publish it.

The tutorial starts by creating a Next.js project and setting up Shadcn UI and Aceternity UI for premade front-end components, with TailwindCSS for styling. Next, you will use Prisma ORM to create data models and generate schemas for Neon's Postgres database. The tutorial also covers file uploads using Uploadcare, Clerk to handle authentication, and creating a billing system with Stripe that allows users to purchase different plans.

You can never overstate the importance of automation in small-scale to large-scale processes. This video buttresses it through its implementation and emphasizes the usability of features like the draggable canvas with a minimap, which allows users to navigate their workflows easily, adapting to the complexity and size of their projects.

5. SaaS AI Chatbot - Email Marketing, Next.js, Clerk, Neon, Uploadcare, Cloudways, Bun, Stripe, Pusher

At this time, it feels more like a disadvantage not to leverage some AI features within your SaaS product. Many software services use AI to improve product features, either as an agent or as a RAG (Retrieval Augmented Generation) embedded feature. This addition provides a great user experience, thus facilitating the better adoption of said product.

Here is a great tutorial on creating an artificially intelligent chatbot that acts as a sales representative. The chatbot is effective in:

Starting a conversation with a website visitor and requesting their email address.
Providing answers based on user responses.
Sending a link to the user, which can take them to a portal page or a payment form, depending on the conversation flow.
Managing conversation flow by sending an email notification to the business owner.

In this tutorial, you’ll add AI into custom domains and embed chatbots into websites. Authenticated users can track domain income, monitor data metrics, and customize chatbot appearance and greetings based on their subscription. The app enables chatbot training by letting business owners set bot training and help desk questions. Additionally, it supports product sales through Stripe integration, allows users to manage billing and subscriptions, schedule appointments manually or via the chatbot, and conduct email marketing based on captured leads.

In this tutorial, you’ll create a Next.js project with TailwindCSS and build custom authentication pages for Clerk without the watermark. This means you’ll create a custom Clerk authentication component, allowing you to have a UI without the Clerk branding in the authentication component. You’ll then set up file uploads using Uploadcare and create custom theming with Shadcn UI for light and dark modes. A real-time chat feature will be built using Pusher as a secure layer between the server and the client.

Toward the end, you’ll integrate OpenAI and use prompt engineering to customize chatbot responses. Neon will serve as the database, with Prisma ORM managing users, domains, chatbots, billing, help desks, and customer interactions.

This tutorial offers a hands-on guide to integrating AI for client interactions. While not entirely beginner-friendly, it demonstrates AI’s ability to manage early conversations and ensure prompt responses. The standout feature is its adaptive AI, which makes clients feel heard and enhances customer relations without the immediate interference of the business owner.

Final Thoughts

From finance management and subscription-based platforms to AI chatbots and automation tools, these five tutorials highlight the variety of directions your SaaS project can take. One feature that stands out across all these tutorials is using Neon PostgreSQL as the database of choice. No matter which technologies or frameworks you adopt, Neon can help you:

Scale on demand without manual intervention
Save time setting up and managing databases
Integrate with TypeScript, Python, Next.js, and other common SaaS stacks

Building a SaaS product doesn’t have to be a giant leap of faith. With guides like these, modern platforms like Next.js and Django, and serverless databases like Neon, you can experiment, iterate, and ship faster.

Ready to start building your own SaaS app? Pick a tutorial, dive in, and take the first step toward launching your product. Whether building from scratch or improving an existing idea, the right tools and guidance can help you turn your vision into reality.

Building AI-powered mobile app with RAG, pgvector & OpenAI

Femi-ige Muyiwa — Wed, 21 Feb 2024 10:34:42 +0000

Retrieval augmented generation (RAG) is an AI technique for enhancing the response capabilities of large language models (LLMs) by providing them with an external source of knowledge. This framework ensures that the model can access current and reliable facts because the users can access the model sources.

In this tutorial, you’ll learn how to implement the RAG technique by building a mobile application that gives a detailed response with respect to an external data source (CSV file) when queried. We’ll also cover how to implement this functionality with Neon Postgres, Flutter framework, and OpenAI.

Check out the link to the GitHub repository containing all the code.

As you begin, this tutorial will explore the following:

Creating a Neon Postgres project, database, and table.
Component of RAG
Building a Flutter application
Connecting Neon to the Flutter application

Prerequisites

We’ll need a few things before getting started:

A good understanding of the Flutter framework and dart programming.
A Neon account.
An OpenAI API key; thus, you need an OpenAI account.

Breaking down the technologies

Let’s talk a bit about the tools we’ll be using. Neon is a fully managed serverless Postgres with a generous free tier that provides separate storage and computing to offer autoscaling, branching, and bottomless storage. Neon is fully open source under the Apache 2.0 licenses, and you can find the neondatabase on GitHub.

We’ll also be using the pgvector Postgres extension on Neon. pgVector is a Postgres extension that works with vector embeddings for storage, similarity search, and more.

Enabling the pgvector extension in your database simplifies storing vector embeddings as well as easy querying using the inner product (<#>) or cosine distance (<=>).

Note: vector embeddings are numerical representations of data that capture semantic relationships and similarities

The project further explores the use case of the RAG technique by employing the Flutter framework. Flutter is a UI development toolkit developed by Google that supports cross-platform app-building using the same codebase. The use of the Flutter application for this project is to give a programmatic and visual understanding of how the RAG technique works. Thus, we will build a simple chatbot that offers a response based on the external data source provided at a time.

Finally, the project comes to life using both technologies by pairing with a Large Language Model (LLM) via OpenAI to process the external data source and deliver a response in natural language.

Some paragraphs above mentioned a mystery phrase called vector embeddings, leading to more questions. The coming sections will demystify this concept by giving a better understanding of the things needed to implement the RAG technique in your application.

Let's begin without further ado.

Setting up a Neon Postgres project

Neon offers a simple method of setting up a project — streamlined to three things:

Project name
Postgres version
Database name

This option is made available after signing into your Neon account. On the project setup page, there are more options to explore; for example, you can change the branch name to any other name, but advisably, leave it as main for now and click Create project.

Now, on the homepage, you’ll find a popup containing all the connection details required to connect your application(s) to a Neon database. Copy these details to a safe file, such as a JSON file.

Note: For security reasons, you should use a .env file to secure your connection details. Check out this article to learn more about how to use flutter_dotenv.

With Neon, you can manage your database using a few options, such as a Command Line Interface (CLI), the Neon API, or a Structured Query Language (SQL) tool. Neon also provided an SQL Editor to write SQL commands directly on the console. You are in luck with SQL commands, as Flutter supports a Dart package Postgres that makes this possible.

The coming sections will shed more light on using Flutter and Neon Postgres pgvector extension and OpenAI to implement RAG.

Creating the Flutter application

Cloning the UI template

The project implements a Flutter template for faster development by making the UI code available using GitHub. To begin, clone the template from the repository using the command below:



git clone https://github.com/muyiwexy/neon_rag_with_openai.git

Note: The template contains the Flutter UI code and some defragmented codes, which will cause some debug errors.

After cloning the project, run the command below to obtain all the dependencies listed in the pubspec.yaml file in the current working directory and their transitive dependencies.:



flutter pub get

We employed the Model View Controller (MVC) architecture to handle specific development aspects of the application and give it a clear separation of concern. The architecture helps maintain readability by separating the business (core) logic from the UI or view (presentation layer). The cloned template won't compile at the moment as some things must be implemented as you progress.

Here’s a helpful ASCII representation of the main Flutter development folder (lib) using the MVC design pattern:



lib/
├─ home/
│  ├─ view/
│  │  ├─ home_page.dart
├─ core/
│  │  ├─ config/
│  │  │ ├─ service_config.dart
│  │  ├─ dependency_injection/
│  │  │ ├─ provider_locator.dart
├─ main.dart

Notes: The intended data source is a comma-separated values (CSV) file called mydata.csv in the assets folder.

The external dart plugins used during the project (they are in the template pubspec.yaml file) :



dependencies:
  csv
  flutter_dotenv
  http
  postgres
  provider

Implementing the RAG technique

Implementing a Flutter project as a RAG application involves the functionalities being classed into two components:

Indexing
Retrieval

Indexing

Indexing allows the system to receive information from an extensive data set by creating means of data collection, encoding, and storage. To successfully index the intended data (mydata.csv), you will need to approach the indexing component in three processes:

Load
Split
Store

Load
The load process is usually offline; thus, with the CSV package, you can convert a CSV file to a list of Strings with the convert method of the CsvToListConverter() class.

To load a CSV file, head to the lib/home directory and create a new controller directory. Within this newly created directory, create a file called openai_indexing_services.dart to handle the abstraction of the business logic from the UI.

The openai_indexing_services.dart will handle all the abstractions for the methods in this project, leading to some back and forth in the coming sections. For the first method, create a function loadCSV() to handle loading the CSV file in this project.



abstract class OpenAIIndexingServices {
  Future<List<List<dynamic>>> loadCSV();
}

Still, within the lib/home/controller directory, create another file called openai_indexing_services_impl.dart to handle the implementation of the abstraction OpenAIIndexingServices above. It does this using inheritance — one of the fundamentals of object-oriented programming (OOP).

The implementation of the loadCSV() function uses the loadString method in the rootbundle property to get the CSV file from its root directory. Then, the convert method in the CsvToListConverter() class (called from the CSV package) converts the CSV document to a list of lists (with each inner list representing a row of the CSV documents) and then returns the value.



class OpenAIIndexingServicesImpl extends OpenAIIndexingServices {

  @override
  Future<List<List<dynamic>>> loadCSV() async {
    debugPrint("Loading File... ");
    final rawcsvFile = await rootBundle.loadString("assets/mydata.csv");
    final csvData =
        const CsvToListConverter().convert(rawcsvFile.splitMapJoin("/n"));
    return csvData;
  }
}
void debugPrint(String message) {
  if (kDebugMode) {
    print(message);
  }
}

Split
Next, you’ll need to split this data into smaller chunks and embed them as vectors to aid the indexing and passing of large data into models, as large data takes too long to search over.

To split and embed the loaded data into chunks, update the abstraction in the lib/home/controller/openai_indexing_services.dart with the code below:



abstract class OpenAIIndexingServices {
  Future<List<List<dynamic>>> loadCSV();
  Future<List<List<String>>> splitToChunks(List<List<dynamic>> csvDoc);
  Future<List<List<double>>> getEmbeddings(List<List<String>> chunks);
}

The splitToChunks method takes in the value returned from the loadCSV (csvDoc) as an argument and returns a list of lists of Strings.

The implementation of the splitToChunks method in the lib/home/controller/openai_indexing_services_impl.dart takes a list of lists (csvDoc) as a parameter, where each inner list in the csvDoc represents a row in a CSV document.

Then, it initializes an empty list chunkList to store chunks of split data ( for later) and iterates over each row in csvDoc using a for loop, starting from the second row (var i = 1), treating each row as a document.

Within the iteration or for loop, specify the index of the content intended to be split, which then gets the content string for every row. After, obtain the ideal size of a chunk through the floor division of a set max idealTokenSize (512) by 1.3 and explicitly convert the subsequent result to an integer.

Next, split the content string gotten earlier into words using a space delimiter (""), filter a list of words, remove occurrences of the delimiter, and assign the resulting list back to the variable. Then, it gets the length of the new list using the length method, which is now the total number of words in the content.

After, it determines the number of chunks needed based on a floor division of the total words by the ideal size of a chunk. The document is split into ideal-sized chunks, and each chunk is added to the chunkList with an ID that combines the original document's ID and the chunk number.

Finally, the splitToChunks method returns the chunkList. The splitToChunks method ensures that each chunk is approximately of the ideal size, but the actual size can vary depending on the word boundaries.



class OpenAIIndexingServicesImpl extends OpenAIIndexingServices {

  // loadCSV

  @override
  Future<List<List<String>>> splitToChunks(List<List<dynamic>> csvDoc) async {
    List<List<String>> chunkList = [];
    for (var i = 1; i < csvDoc.length; i++) {
      String content = csvDoc\[i\][1];
      int start = 0;
      int idealTokenSize = 512;
      int idealSize = (idealTokenSize ~/ (4 / 3)).toInt();
      int end = idealSize;
      List<String> words = content.split(" ");
      words = words.where((word) => word != " ").toList();
      int totalWords = words.length;
      int chunks = totalWords ~/ idealSize;
      if (totalWords % idealSize != 0) {
        chunks += 1;
      }
      List<String> newContent = [];
      for (int j = 0; j < chunks; j++) {
        if (end > totalWords) {
          end = totalWords;
        }
        newContent = words.sublist(start, end);
        String newContentString = newContent.join(" ");
        String id = csvDoc\[i\][0];
        chunkList.add(["${id}_$j", newContentString]);
        start += idealSize;
        end += idealSize;
      }
    }
    return chunkList;
  }
}
void debugPrint(String message) {
  if (kDebugMode) {
    print(message);
  }
}

To embed the chunkList, implement the getEmbeddings method and pass the chunkList as an input. Here is the code below:



class OpenAIIndexingServicesImpl extends OpenAIIndexingServices {
  late final http.Client client;
  OpenAIIndexingServicesImpl({
    required this.client,
  });

  // loadCSV

  // splitToChunks

  @override
  Future<List<List<double>>> getEmbeddings(List<List<String>> chunks) async {
    debugPrint("Embedding ....");
    List<List<double>> embeddedDoc = [];
    for (var chunk in chunks) {
      final response = await client.post(
        Uri.parse("https://api.openai.com/v1/embeddings"),
        headers: {
          'Content-Type': 'application/json',
          'Authorization': 'Bearer ${dotenv.env['OPENAI_API_KEY']!}',
        },
        body: jsonEncode({
          'model': 'text-embedding-ada-002',
          'input': chunk[0],
        }),
      );
      if (response.statusCode == 200) {
        final Map<String, dynamic> responseData = jsonDecode(response.body);
        List<dynamic> embedding = responseData\['data'\][0]['embedding'];
        List<double> embeddingdouble = embedding.map((item) {
          if (item is double) {
            return item;
          } else {
            throw const FormatException('Invalid format');
          }
        }).toList();
        embeddedDoc.add(embeddingdouble);
      } else {
        throw response.body;
      }
    }
    debugPrint("Embedding complete....");
    return embeddedDoc;
  }
}
void debugPrint(String message) {
  if (kDebugMode) {
    print(message);
  }
}

The getEmbeddings method takes the chunkList list — a list of strings as an input. It will iterate over every list item within the chunkList list and perform a specific action to create embeddings for them.

To achieve this, start by initializing an empty list of lists of doubles embeddedDoc to store the embeddings. Then, use a for loop to iterate over each list item in the chunkList list while performing the following actions on each iteration:

Using the http package, make a POST request to the OpenAI API to get the embedding of a list item from the chunkList list.
If the response status is 200, decode the response body to get the embedding, convert it to a list of doubles, and add it to embeddedDoc.
If the response status is not 200, throw an exception with the response body as the message.

Finally, let the getEmbeddings method return embeddedDoc, a list of embeddings where each embedding is a list of doubles.

Store
Now, you have a couple of data (chunkList and embeddedDoc) that need storing in someplace, and that is where your Neon database comes in. To store your embedding (embeddedDoc) and chunks (chunkList), you must create a database table and enable the pgvector's vector extension in the database. The vector extension is handy as it lets you search using a vector cosine's similarity index when querying the database later.

Thus, start by updating the OpenAIIndexingServices abstraction in the lib/home/controller/openai_indexing_services.dart file with the code below:



abstract classOpenAIIndexingServices {
  // the abstraction above
  Future<bool> checkExtExist();
  Future<bool> checkTableExist(String tableName);
  Future<String> createNeonVecorExt();
  Future<String> createNeonTable(String tableName);
  Future<String> deleteNeonTableRows(String tableName);
  Future<void> storeDoument(List<List<String>> chunks,
      List<List<double>> embeddedVectors, String tableName);
}

Remember, Neon allows database management using SQL commands — equally, you can execute these SQL commands programmatically in Flutter using the Postgres package.

Thus, for the implementation in the lib/home/controller/openai_indexing_services_impl.dart, the checkExtExist, and checkTableExist methods are condition checkers that use the SELECT statement to confirm if the vector extension and a particular table exist within the database. They return boolean values dependent on the result obtained from the request.

Next, use the createNeonVecorExt method to create a 'vector' extension in the Neon Postgres database. Also, the createNeonTable method creates a specified table in the Neon Postgres database with id, metadata, and embedding fields. It also creates an index on the embedding field using the ivfflat algorithm and vector_cosine_ops operator class. The deleteNeonTableRows method truncates all rows from a specified table in the Neon Postgre database.

Finally, the storeDoument method stores the data in the Neon database table created earlier by iterating over the chunkList and embeddedDoc, inserting the id, metadata, and embedding into the table.

The metadata field is a map containing pageContent and txtPath keys, which are the IDs in the chunkList, while the embedding field is a string representation of an array of doubles.



class OpenAIIndexingServicesImpl extends OpenAIIndexingServices {
  late final http.Client client;
  late final Connection connection;
  OpenAIIndexingServicesImpl({
    required this.client,
    required this.connection,
  });

  // loadCSV

  // splittoChunks

  // getEmbeddings

  @override
  Future<bool> checkExtExist() async {
    final checkExtExist = await connection.execute(
      "SELECT EXISTS (SELECT FROM pg_extension WHERE extname = 'vector');",
    );
    return checkExtExist.first[0] as bool;
  }

  @override
  Future<bool> checkTableExist(String tableName) async {
    final checkTableExist = await connection.execute(
      "SELECT EXISTS (SELECT FROM information_schema.tables WHERE  table_schema = 'public' AND table_name = '$tableName');",
    );
    return checkTableExist.first[0] as bool;
  }

  @override
  Future<String> createNeonVecorExt() async {
    debugPrint("Creating pgvector extension ...");
    await connection.execute("CREATE EXTENSION vector;");
    return "Vector extension created Successfully";
  }

  @override
  Future<String> createNeonTable(String tableName) async {
    debugPrint("Creating the $tableName table ... ");
    await connection.execute(
      "CREATE TABLE $tableName (id text, metadata text, embedding vector(1536));",
    );
    debugPrint("Indexing the $tableName using the ivfflat vector cosine");
    await connection.execute(
        'CREATE INDEX ON $tableName USING ivfflat (embedding vector_cosine_ops) WITH (lists = 24);');
    return "Table created successfully";
  }
  @override
  Future<String> deleteNeonTableRows(String tableName) async {
    debugPrint("Deleting tableRows");
    await connection.execute("TRUNCATE $tableName;");
    return "Table rows deleted successfuly";
  }

  @override
  Future<void> storeDoument(
    List<List<String>> chunks,
    List<List<double>> embeddedVectors,
    String tableName,
  ) async {
    debugPrint("Storing data... ");
    await connection.runTx((s) async {
      for (int i = 0; i < chunks.length; i++) {
        final chunk = chunks[i];
        final embeddingArray = embeddedVectors[i];
        await s.execute(
            Sql.named(
              'INSERT INTO $tableName (id, metadata, embedding) VALUES (@id, @metadata, @embedding)',
            ),
            parameters: {
              'id': chunk[0],
              'metadata': {
                'pageContent': chunk[1],
                'txtPath': chunk[0],
              },
              'embedding': '$embeddingArray',
            });
      }
    });
    debugPrint("Data stored!!!");
  }
}
void debugPrint(String message) {
  if (kDebugMode) {
    print(message);
  }
}

With this, you have successfully indexed the data; the next step is to retrieve it!

Retrieval

Retrieval is a streamlined component commonly divided into two processes:

Retrieve: This is done by comparing the vector embedding of a user query with the closest available result present in the database. We perform this comparison using the cosine similarity search to compare a vector with another. Thus, when you get the closest results, you can use it for the second process.
Generate: After getting the closest result, you can use it as an LLM assistance to generate responses based on that information.

To do this, head to the OpenAIIndexingServices abstract class in the lib/home/controller/openai_indexing_services.dart and update it with the code below:



abstract class OpenAIIndexingServices {
  // do something
  Future<String> queryNeonTable(String tableName, String query);
}

The method above returns a string response by following the retrieval process above.

Here is the code for the implementation of the queryNeonTable method below:

openai_indexing_service_impl.dart

For the OpenAIIndexingServicesImpl class in the code above, you must define a method getCompletionFromMessages that:

Makes a POST request to the OpenAI API to get a completion from a list of messages.
If the response status is 200, it decodes the response body to get the completed content.
If the response status is not 200, it throws an exception with the response body as the message.

Next, define a method getQueryEmbeddings that:

Makes a POST request to the OpenAI API to get the embedding of a query.
If the response status is 200, it decodes the response body to get the embedding, converts it to a list of doubles, and returns it.
If the response status is not 200, it throws an exception with the response body as the message.

Finally, use the queryNeonTable method to get the embedding of a query by calling the getQueryEmbeddings method. Afterward, execute an SQL query on the connection using Postgres to get similar items from a specified table. After getting a list of similar items from the table, convert the result into a list of Metadata objects.

Add a condition to check if Metadata is not empty, and if valid, concatenate the page content, make a list of messages, get a completion from the messages, and return it. If Metadata is empty, it returns a default message: "Couldn't find anything on that topic."

Here is the Metadata objects model class below:



class Metadata {
  String pageContent;
  String txtPath;
  Metadata({
    required this.pageContent,
    required this.txtPath,
  });
  factory Metadata.fromJson(Map<String, dynamic> json) => Metadata(
        pageContent: json["pageContent"],
        txtPath: json["txtPath"],
      );
  Map<String, dynamic> toJson() => {
        "pageContent": pageContent,
        "txtPath": txtPath,
      };
}

Note: Save it in the lib/home/model/ directory

Handling the abstraction, state management, and dependency injection

Though you have created and implemented functionalities that encapsulate the idea of this project, it is still all business logic. This means there is no connection between the business logic and the presentation layer (UI). To implement this connection, there will need to be a "handshake" between these layers, and you can do this by creating a class that will have direct contact with the presentation layer.

To begin, create a new directory called view_model in the lib/home/ folder, this folder will have two files to handle indexing and retrieval, respectively. Create a file in the lib/home/view_model/ directory called index_notifier.dart. This file will cater to the indexing processes established earlier and the application's state involved in the process.

The project will explore a simple management solution called ValueNotifiersfor state management. This is mainly down to the fact that you aren't dealing with a complex.
UI and might be listening to single values throughout the project.

So, start by creating an enum class within the lib/home/view_model/index_notifier.dart file with initial, loading, loaded, and error values to track the application’s state. Then, create an IndexNotifier class and instantiate the OpenAIIndexingServices object from the lib/home/controller/openai_indexing_services.dart. Make sure to pass in the OpenAIIndexingServices object to the constructor.

Next, define a ValueNotifier, for indexState to track the state of the indexing process, followed by creating a method indexingNeonTable. The indexingNeonTable does the following:

Sets indexState to loading.
Loads a CSV document using openAIIndexingServices.
Splits the document into chunks and gets their embeddings.
Check if the 'vector' extension exists in the database and create it if not.
Check if a specified table exists in the database and create it if not. If the table already exists, it deletes all rows from the table.
Stores the chunks and their embeddings in the table.
Sets indexState to loaded.
Handles errors by setting indexState to error and rethrowing the exception.
After a delay, set indexState to initial.

Here is the code in the index_notifier.dart below:



enum IndexState { initial, loading, loaded, error }

class IndexNotifier {
  late final OpenAIIndexingServices openAIIndexingServices;
  IndexNotifier({required this.openAIIndexingServices});

  final _indexState = ValueNotifier<IndexState>(IndexState.initial);
  ValueNotifier<IndexState> get indexState => _indexState;

  indexingNeonTable() async {
    try {
      _indexState.value = IndexState.loading;
      final csvDoc = await openAIIndexingServices.loadCSV();
      final chunks = await openAIIndexingServices.splitToChunks(csvDoc);
      final embeddedDocs = await openAIIndexingServices.getEmbeddings(chunks);

      if (!(await openAIIndexingServices.checkExtExist())) {
        await openAIIndexingServices.createNeonVecorExt();
      }
      if (!(await openAIIndexingServices
          .checkTableExist(ServiceConfigurations.openAITable))) {
        await openAIIndexingServices
            .createNeonTable(ServiceConfigurations.openAITable);
      } else {
        await openAIIndexingServices
            .deleteNeonTableRows(ServiceConfigurations.openAITable);
      }

      await openAIIndexingServices.storeDoument(
        chunks,
        embeddedDocs,
        ServiceConfigurations.openAITable,
      );
      _indexState.value = IndexState.loaded;
    } catch (e) {
      _indexState.value = IndexState.error;
      rethrow;
    } finally {
      await Future.delayed(const Duration(milliseconds: 2000));
      _indexState.value = IndexState.initial;
    }
  }
}

Create another file within the lib/home/view_model called query_notifier.dartfor the second file. This file does similarly to the index_notifier.dart file.

Begin by creating a Message model class with String values query and response and pass them into the constructor. Also, create an enum class within the file with values initial, loading, loaded, error to track the state of the application when a user queries the LLM.

Create a QueryNotifier class that initializes the OpenAIIndexingServices object from the lib/home/controller/openai_indexing_services.dart, followed by defining a ValueNotifier for _queryandResponseState to track the state of the query and response.

After, define a ValueNotifier for queryState to track the state of the query. Follow this by creating a method queryIngeonTable that does the following:

Adds a new Message to _messages.
Sets queryState to loading.
Calls queryNeonTable method of openAIIndexingServices to get a response.
Updates the last message's response and sets queryState to loaded.
Handles errors by setting queryState to error, then back to initial after a delay.



class QueryandResponse {
  String? query;
  String? response;
  QueryandResponse({required this.query, this.response = ""});
}
enum QueryState {
  initial,
  loading,
  loaded,
  error,
}
class QueryNotifier {
  late final OpenAIIndexingServices openAIIndexingServices;
  QueryNotifier({required this.openAIIndexingServices});
  final List<QueryandResponse> _queryandResponse = [];
  final _queryandResponseState = ValueNotifier<List<QueryandResponse>>([]);
  ValueNotifier<List<QueryandResponse>> get queryandResponseState =>
      _queryandResponseState;

  final _queryState = ValueNotifier<QueryState>(QueryState.initial);
  ValueNotifier<QueryState> get queryState => _queryState;

  queryIngeonTable(String query) async {
    try {
      _queryandResponse.add(QueryandResponse(query: query, response: ""));

      _queryandResponseState.value = List.from(_queryandResponse);
      _queryState.value = QueryState.loading;

      String response = await openAIIndexingServices.queryNeonTable(
          ServiceConfigurations.openAITable, query);

      final List<QueryandResponse> updatedMessages =
          List.from(_queryandResponse);
      updatedMessages.last.response = response;

      _queryandResponseState.value = updatedMessages;
    } catch (e) {
      // Handle errors if necessary
      print(e);
      _queryState.value = QueryState.error;
      await Future.delayed(const Duration(milliseconds: 2000));
      _queryState.value = QueryState.initial;
    }
  }
}

With that, a "handshake" is possible but not yet established, as the Flutter application is unaware of a few things, such as where/what the Postgres connection is established. This is because the Postgres connection is a required field of the OpenAIIndexingServicesImpl class in the lib/home/controller/openai_indexing_services_impl.dart file.

Thus, the next section will clarify how to mount dependencies on the widget tree and connect Flutter to the neon database.

Note: Check out the lib/core/dependency_injection/provider_locator.dart file to see how to inject the objects as a dependency using the provider package.

Connecting Neon database to Flutter application and dependency injection
You used the Postgres package to execute SQL commands during this project. The Postgres package also connects the Flutter app to the Neon database to facilitate the execution of those commands declared in your code. The code containing the connection process is in the provider_locator.dart file from the template.

Here is the code below:



class ProviderLocator {
  // provider tree
  static Future<MultiProvider> getProvider(Widget child) async {
    final openAIIndexingService = await _createOpenAIIndexingServices();
    return MultiProvider(
      providers: [
        Provider<OpenAIIndexingServices>.value(value: openAIIndexingService),
        Provider(
          create: (_) => IndexNotifier(
            openAIIndexingServices: openAIIndexingService,
          ),
        ),
        Provider(
            create: (_) =>
                QueryNotifier(openAIIndexingServices: openAIIndexingService))
      ],
      child: child,
    );
  }
  static Future<OpenAIIndexingServices> _createOpenAIIndexingServices() async {
    final connection = await createPostgresConnection();
    final client = await _createHtttpClient();
    return OpenAIIndexingServicesImpl(connection: connection, client: client);
  }
  // postgres connection
  static Future<Connection> createPostgresConnection() async {
    const maxRetries = 3;
    for (var retry = 0; retry < maxRetries; retry++) {
      try {
        final endpoint = Endpoint(
          host: dotenv.env['PGHOST']!,
          database: dotenv.env['PGDATABASE']!,
          port: 5432,
          username: dotenv.env['PGUSER']!,
          password: dotenv.env['PGPASSWORD']!,
        );
        final connection = await Connection.open(
          endpoint,
          settings: ConnectionSettings(
            sslMode: SslMode.verifyFull,
            connectTimeout: const Duration(milliseconds: 20000),
          ),
        );
        if (connection.isOpen) {
          if (kDebugMode) {
            print("Connection Established!");
          }
          return connection;
        }
      } catch (e) {
        if (kDebugMode) {
          print('Error creating PostgreSQL connection: $e');
        }
      }
      await Future.delayed(const Duration(seconds: 2));
    }
    // If maxRetries is reached and the connection is still not open, throw an exception
    throw Exception(
        'Failed to establish a PostgreSQL connection after $maxRetries retries');
  }
  static Future<http.Client> _createHtttpClient() async {
    try {
      return http.Client();
    } catch (e) {
      rethrow;
    }
  }
}

In the code above, the ProviderLocator class in the code does the following:

Defines a method getProvider that:
- Creates an OpenAIIndexingServices instance.
- Returns a MultiProvider with an OpenAIIndexingServices provider and two other providers for IndexNotifier and QueryNotifier.
Defines a method _createOpenAIIndexingServices that:
- Creates a PostgreSQL connection using the connection details from the Neon database.
- Creates an HTTP client.
- Returns an OpenAIIndexingServicesImpl instance with the created connection and client.
Defines a method createPostgresConnection that:
- Tries to establish a PostgreSQL connection with specified settings.
- If the connection fails, it retries up to a maximum number of times.
- If the connection is not established after maximum retries, it throws an exception.
Defines a method _createHtttpClient that returns an HTTP client.

With that, you have successfully mounted the database on the widget tree and established a connection between the Neon Postgres database and Flutter. When you run the application, you should have your results as shown in the video below:

rag with openai | Opentape

Muyiwa Femi-Ige - Feb 21st, 10:32am

app.opentape.io

Conclusion

This tutorial journeyed into the world of RAG and its implementation in mobile applications using the Flutter framework, Neon Postgres, and OpenAI. It explored how RAG enhances the response capabilities of LLMs by providing them with an external knowledge source.

You also learned about Neon and its capabilities with the pgvector extension. If you’ve gotten this far, you’re now equipped with the knowledge to build more intelligent and responsive applications using RAG, opening up new possibilities in AI.

If this piqued your curiosity, let's contribute to advancing AI, making it more accessible, efficient, and beneficial for all. Happy coding!

Resources

Here are some resources that will guide you more in this journey:

How to set up Auth0 and Neon Postgres in Next.js App Router

Femi-ige Muyiwa — Tue, 13 Feb 2024 09:34:54 +0000

Modern applications demand a balance of scalability, speed, and security to meet users’ ever-growing needs. Ensuring authentication and a robust database management system is crucial at the foundation of this triad. In this article, we’ll talk about the combination of Auth0 and Neon Postgres when you need to accommodate these two needs.

Auth0 is an identity platform that simplifies authentication and authorization functionalities across web, mobile, and native applications.

Neon is fully managed serverless Postgres that provides separate storage and computing to offer autoscaling, branching, and bottomless storage. Neon is open source under the Apache 2.0 licenses; you can check out the neondatabase GitHub repo.

This guide provides a comprehensive walk-through of setting up Auth0 with a Postgres database like Neon by building a login page that routes to a profile page while implementing Neon Postgres as the data store for user information. This application uses the Next.js React framework with App Router and Tailwind for styling.

Here is the link to the GitHub repository containing all the code

Prerequisites

To successfully complete this guide, you’ll need a few things:

Proficiency with JavaScript frameworks such as Next.js (recommended), React, etc.
A Neon account
An Auth0 account
Node 18 or later versions

Setting up a Neon Postgres project

After signing up, you’ll note Neon's streamlined project setup; you’ll just need to provide three things:

Project name
Postgres version
Database name

On the project setup page, there are more options to explore; for example, you can change the branch name to any other name, but we recommend leaving it as main for now. Next, click Create project.

On the homepage, you'll find a popup containing all the connection details required to connect your application(s) to a Neon database. Copy these details to a safe file, such as a JSON file.

Creating a Next.js application and installing dependencies

To create a Next.js application, open your terminal in any folder of your choice and type the command below:

npx create-next-app@latest <folder-name> --ts

This command creates a Next.js application with TypeScript as the preferred programming language.

Next, we need to install some dependencies. These include an Auth0 dependency and dotenv to store sensitive credentials in a .env file. To do this, run the command below:

npm install @auth0/nextjs-auth0 dotenv

Configuring Auth0 application settings

After signing into the Auth0 account you created earlier, click Create Application in the Auth0 dashboard, select Regular Web Applications, and fill in the app name to create traditional web applications that use redirects.

The following prompt asks what technology is being used for the project; we’ll select Next.js out of all the presented options for this project.

Over at your just-created Regular Web Applications, you’ll need to configure the URLs for your app under the application URI. To do this, head to Settings > Allowed Callback URLs and type in the URL below within the input field:

http://localhost:3000/api/auth/callback

Here, we use port 3000 as the port our application runs on, so make sure to select the port your application is running on.

Note: you can add multiple URLs by separating them with a comma ","

The URL above completes the web application's authentication flow by triggering the callback route after successful authentication. The callback route api/auth/callback is integrated by Auth0 into the @auth0/nextjs-auth0 package installed earlier, which will handle the callback flow.

Next, we’ll add the following URL in the Allowed Logout URLs input field:

http://localhost:3000

Setting this allows a redirect back to our web application after logging out, and you can use a personal URL if your project requires it.

Once you're done with that, save the changes and head to the top page of the Settings section. Make sure to note the Domain, Client ID, and Secret, as they will come in handy in the coming sections.

Connecting NextJs to Auth0

To initiate a connection between Auth0 and Next.js, head back to the Next.js application we created and create a .env.local file in the root folder to store the Domain, Client ID, and Secret from Auth0. Then, paste the following variables into the file below:

AUTH0_SECRET= use [openssl rand -hex 32] to generate a 32 bytes value
AUTH0_BASE_URL= http://localhost:3000
AUTH0_ISSUER_BASE_URL= https://AUTH0-DOMAIN
AUTH0_CLIENT_ID= AUTH0-CLIENT-ID
AUTH0_CLIENT_SECRET= AUTH0-CLIENT-SECRET

Note: You can get these pages from Auth0's Nextjs documentation and replace the values with those of your Auth0 regular web app's values.mk

We will need to generate a 32-byte random value for the AUTH0_SECRET, and this can be done using Node's crypto module. The crypto module generates an arbitrary hexadecimal string of length 32. Thus, head to your terminal and type the command below to use the module:

node -e "console.log(crypto.randomBytes(32).toString('hex'))"

Creating an API route for Auth0

Next, we'll need to add the dynamic API route; we’ll do so by creating a /api directory within the /app directory. Within the now /app/api directory, create an /auth directory to handle all the authentication-related things within this project (create this directory instead if you already have an /api).

Within the /auth directory, create a new unique directory called /[auth0] and create a file route.tsx within the folder. Using a directory in this manner, /[auth0] indicates that the directory name auth is a dynamic segment.

In the route.tsx file, we will use the handleAuth method from the @auth0/nextjs-auth0 to handle the authentication flow of our web application. The handleAuth creates multiple route handlers under the hood — this means that when we log initially into Auth0, the routes followed are api-auth-login. So, the handleAuth function will handle all the login flow based on that and log out the callback URLs set earlier.

So, let’s add the code below to the route.tsx to implement what we just explained above:

import { handleAuth } from "@auth0/nextjs-auth0";
export const GET = handleAuth();

Next, we need to wrap our application in an Auth0 UserProvider. In the layout.tsx file from the base /app directory, we’ll import UserProvider from @auth0/nextjs-auth0/client, and then we'll wrap the body tag in a UserProvider.

So here is the updated layout.tsx file below:

import type { Metadata } from "next";
import { Inter } from "next/font/google";
import "./globals.css";
import { UserProvider } from "@auth0/nextjs-auth0/client";
const inter = Inter({ subsets: ["latin"] });
export const metadata: Metadata = {
  title: "Auth0 and Neon Postgres",
  description: "Generated by create next app",
};
export default function RootLayout({
  children,
}: {
  children: React.ReactNode,
}) {
  return (
    <html lang="en">
      <UserProvider>
        <body className={inter.className}>{children}</body>
      </UserProvider>
    </html>
  );
}

We perform this operation using an Auth0 platform to create and authenticate users. Therefore, we will have to wrap our entire application in Auth0. At the moment, these are all the configurations needed for our web application. In the coming sections, we will create pages for logging in and a profile page to display certain user information.

Building the pages and routes using Next.js and Auth0

This project will use the initial page (/app/page.tsx) and profile routes. The initial page route will be a form with email, password input, and a submit button. Also, there will be a button that triggers a connection to Auth0. For the profile area, a simple card will show the user's profile image, name, and email. The card will also contain a logout button that triggers a logout action from Auth0.

The idea is to give a clearer picture of how the App Router operates after successful authentication with Auth0. Earlier, we enabled App Router when we created our project — thus, route by creating another folder in the /app directory, let's call it profile. Within this profile directory, create a file called page.tsx. With that, you've successfully created a route with App Router.

Head back to the /app/page.tsx file, clear out the initial template, and replace it with the one below:

export default function Login() {
  return (
    <main className="flex min-h-screen items-center justify-center">
      <div className="w-[300px] md:w-[600px] shadow-lg p-5">
        <p className="text-5xl">Login</p>
        <div className="h-[20px]" />
        <form>
          <input
            type="email"
            placeholder="E-mail"
            className="border-2 border-black w-full p-5 text-2xl rounded-lg focus:outline-blue-400"
            name="email"
          />
          <div className="h-[20px]" />
          <input
            type="password"
            placeholder="Password"
            className="border-2 border-black w-full p-5 text-2xl rounded-lg focus:outline-blue-400"
            name="password"
          />
          <div className="h-[40px]" />
          <div className="flex justify-end">
            <button className="border-2 border-black p-5 px-10 text-2xl rounded-lg">
              Try Me
            </button>
          </div>
        </form>
        <div className="h-[20px]" />
        <p className="flex justify-center font-bold text-2xl">Or</p>
        <div className="h-[20px]" />
        <button className="border-2 w-full border-black p-5 px-10 text-2xl rounded-lg">
          Sign in with Auth0
        </button>
      </div>
    </main>
  );
}

The code above is the login UI template mentioned earlier, and when we run the code, our application will look like the image below:

Still within the /app/pages.tsx file, import useUser function from @auth0/nextjs-auth0/client. Then return user, error, isLoading from the useUser function in the Login function. The user object allows us to display user information after a successful login. In this case, where we experience any errors during signups or login, the error object will display the error message. isLoading is a Boolean parameter that lets us know when data is being loaded.

Also, we will use Next.js navigation to handle navigation using the useRouter function. To do this, we’ll import the useRouter function from next/navigation, then return push.

With this, we can use a condition to check if there is a user object, and if it is true, push it to the /profile route created earlier. Also, we will use the useRouter function to create a login link that will allow us to sign up and log in to our Auth0 application.

First, create a function that pushes to the route api/auth/login and set an onClick event to the function in the Sign in with Auth0 button from our template code.

Below is the updated code for the page.tsx:

"use client";
import { useRouter } from "next/navigation";
import { useUser } from "@auth0/nextjs-auth0/client";
export default function Login() {
  const { push } = useRouter();
  const { user, error, isLoading } = useUser();
  if (user) {
    push("/profile");
  }
  if (isLoading) {
    return <div>...loading</div>;
  }
  if (error) {
    return <div>{error.message}</div>;
  }
  const handleLogin = () => push("api/auth/login");
  return (
    <main className="flex min-h-screen items-center justify-center">
      <div className="w-[300px] md:w-[600px] shadow-lg p-5">
        <p className="text-5xl">Login</p>
        <div className="h-[20px]" />
        <form>
          <input
            type="email"
            placeholder="E-mail"
            className="border-2 border-black w-full p-5 text-2xl rounded-lg focus:outline-blue-400"
            name="email"
          />
          <div className="h-[20px]" />
          <input
            type="password"
            placeholder="Password"
            className="border-2 border-black w-full p-5 text-2xl rounded-lg focus:outline-blue-400"
            name="password"
          />
          <div className="h-[40px]" />
          <div className="flex justify-end">
            <button className="border-2 border-black p-5 px-10 text-2xl rounded-lg">
              Try Me
            </button>
          </div>
        </form>
        <div className="h-[20px]" />
        <p className="flex justify-center font-bold text-2xl">Or</p>
        <div className="h-[20px]" />
        <button
          onClick={handleLogin}
          className="border-2 w-full border-black p-5 px-10 text-2xl rounded-lg"
        >
          Sign in with Auth0
        </button>
      </div>
    </main>
  );
}

When you click the button above, an Auth0 login will be created successfully, but the objective is incomplete as there is currently nothing to display in the /profile/page.tsx. So, we’ll add a UI template code that contains a user image area, name, email, and a logout button.

Thus, head to the /profile/page.tsx file and paste the template code below to it:

export default function Profile() {
  return (
    <main className="max-w-[1900px] flex flex-col mx-auto mt-5 rounded-2xl overflow-hidden">
      <div className="h-[250px] bg-black relative">
        <div className="h-[150px] w-[150px] rounded-full border-2 bg-gray-600 absolute -bottom-[75px] z-50 left-10 overflow-hidden">
          <img
            src={user?.picture ?? "default-image-url"}
            alt=""
            className="object-cover h-full, w-full"
          />
        </div>
      </div>
      <div className="h-[300px] relative transl bg-slate-600 p-5 flex justify-between">
        <div>
          <p className="mt-[90px] text-3xl font-bold">
            {user?.name ?? "default-name"}
          </p>
          <p className="mt-[20px] text-2xl">{user?.email ?? "default-email"}</p>
        </div>
        <div>
          <button className="border-2 border-black p-5 px-10 text-2xl rounded-lg cursor-pointer">
            Log Out
          </button>
        </div>
      </div>
    </main>
  );
}

When we run the code, the /profile route should look like the image below:

Still within the /app/profile/page.tsx file, we'll do similarly to what we did in the /app/pages.tsx file with a slight twist. Start by importing the useUser function and the withPageAuthRequired function from @auth0/nextjs-auth0/client. Then return user, error, isLoading from the useUser function in the Profile function.

Next, use the useRouter function to create a logout link to allow us to log out of our Auth0 application. To do this, create a function that pushes to the route api/auth/logout and set an onClick event to the function in the Log Out button from the /profile route template code.

The withPageAuthRequired is a higher-order component used to protect pages requiring authentication. So, we will need to wrap the Profile function with a map object that contains properties onRedirecting and onError that shows specific messages per the context.

Here is the updated code to the /app/profile/page.tsx file:

"use client";
import { useUser, withPageAuthRequired } from "@auth0/nextjs-auth0/client";
import { useRouter } from "next/navigation";
function Profile() {
  const { user, error, isLoading } = useUser();
  const { push } = useRouter();
  // arrow function that routes to the logout api from the [handleAuth] method
  const handleLogout = () => push("api/auth/logout");
  // handle isLoading state from Auth0
  if (isLoading) {
    return <div>...loading</div>;
  }
  // handle error state from Auth0
  if (error) {
    return <div>{error.message}</div>;
  }
  console.log(user?.email);

  // main profile pagelayout
  return (
    <main className="max-w-[1900px] flex flex-col mx-auto mt-5 rounded-2xl overflow-hidden">
      <div className="h-[250px] bg-black relative">
        <div className="h-[150px] w-[150px] rounded-full border-2 bg-gray-600 absolute -bottom-[75px] z-50 left-10 overflow-hidden">
          <img
            src={user?.picture ?? "default-image-url"}
            alt=""
            className="object-cover h-full, w-full"
          />
        </div>
      </div>
      <div className="h-[300px] relative transl bg-slate-600 p-5 flex justify-between">
        <div>
          <p className="mt-[90px] text-3xl font-bold">{user?.name}</p>
          <p className="mt-[20px] text-2xl">{user?.email}</p>
        </div>
        <div>
          <button
            onClick={handleLogout}
            className="border-2 border-black p-5 px-10 text-2xl rounded-lg cursor-pointer"
          >
            Log Out
          </button>
        </div>
      </div>
    </main>
  );
}
export default withPageAuthRequired(Profile, {
  onRedirecting: () => <div>Redirecting to Login ....</div>,
  onError: (error) => <div>{error.message}</div>,
});

With this, we have reached a milestone, as when we run our application with the command npm run dev, we can authenticate our application using Auth0.

auth0 next js | Opentape

Muyiwa Femi-Ige - Feb 13th, 9:26am

app.opentape.io

You’re likely wondering where this data is stored after logging in or signing up. Auth0 utilizes its database to store the data, but we'd like to change that by implementing Auth0 with a custom database — in this case, Neon Postgres. This process ensures user data is available in the primary data store (Neon) for other application development purposes.

The next section demonstrates how to use Neon as a custom database and sets up some database action scripts for direct communication between Auth0 and Neon Postgres. Let’s jump in!

Setting up Neon as a custom database on Auth0

Head back to your Auth0 dashboard, and from there, select Authentication → Database.

In the Database section, create a database connection by clicking Create DB Connection, fill in your desired Connection name, and click Create.

In your newly created connection, head to the Custom Database section, where all the actions will take place. Toggle the Use my own database option and scroll to the Database Action Scripts area.

In the Database Action Scripts section, we can create custom serverless scripts that perform specific actions on our database. The Login script is mandatory; you can set the others as you please. For this tutorial, we will create templates for the six scripts in the image above, and here is their code below in succession:

Login script

This script is executed during each login to validate the user's authenticity.

function login(email, password, callback) {
  const { Pool } = require("pg");
  const bcrypt = require("bcrypt");

  const pool = new Pool({
    connectionString: configuration.POSTGRES_URL,
  });

  const query =
    "SELECT id, nickname, email, password FROM users WHERE email = $1";
  pool.query(query, [email], function (err, result) {
    if (err || result.rows.length === 0)
      return callback(err || new WrongUsernameOrPasswordError(email));

    const user = result.rows[0];

    bcrypt.compare(password, user.password, function (err, isValid) {
      if (err || !isValid)
        return callback(err || new WrongUsernameOrPasswordError(email));

      return callback(null, {
        user_id: user.id,
        nickname: user.nickname,
        email: user.email,
      });
    });
  });
}

Create script

This script is executed after a user signs up. A database table row is created for the user with the information filled in during the signup process.

function create(user, callback) {
  const { Pool } = require("pg");
  const bcrypt = require("bcrypt");

  const pool = new Pool({
    connectionString: configuration.POSTGRES_URL,
  });

  bcrypt.hash(user.password, 10, function (err, hashedPassword) {
    if (err) return callback(err);

    const query =
      "INSERT INTO users(nickname, email, password) VALUES ($1, $2, $3)";
    pool.query(
      query,
      [user.username, user.email, hashedPassword],
      function (err, result) {
        if (err) return callback(err);

        return callback(null);
      }
    );
  });
}

Verify script

This script verifies the email verification status of a user.

function verify(email, callback) {
  const { Pool } = require("pg");
  const pool = new Pool({
    connectionString: configuration.POSTGRES_URL,
  });

  const query =
    "UPDATE users SET email_verified = true WHERE email_verified = false AND email = $1";
  pool.query(query, [email], function (err, result) {
    if (err) return callback(err);

    return callback(null, result && result.rowCount > 0);
  });
}

Change password script

This script changes the user's encrypted password on the database after following the change password during a password reset process.

function changePassword(email, newPassword, callback) {
  const { Pool } = require("pg");
  const bcrypt = require("bcrypt");

  const pool = new Pool({
    connectionString: configuration.POSTGRES_URL,
  });

  bcrypt.hash(newPassword, 10, function (err, hash) {
    if (err) return callback(err);

    const query = "UPDATE users SET password = $1 WHERE email = $2";
    pool.query(query, [hash, email], function (err, result) {
      if (err) return callback(err);

      return callback(err, result && result.rowCount > 0);
    });
  });
}

Get user script

This script tests if a user exists in a database. For example, a scenario in which the user wants to change their password.

function loginByEmail(email, callback) {
  const { Pool } = require("pg");

  const pool = new Pool({
    connectionString: configuration.POSTGRES_URL,
  });

  const query = "SELECT id, nickname, email FROM users WHERE email = $1";
  pool.query(query, [email], function (err, result) {
    if (err || result.rows.length === 0) return callback(err);

    const user = result.rows[0];

    return callback(null, {
      user_id: user.id,
      nickname: user.nickname,
      email: user.email,
    });
  });
}

Delete script

This Script is activated when the user is deleted from Auth0 — thus, the user data is subsequently removed from the database.

function remove(id, callback) {
  const { Pool } = require("pg");

  const pool = new Pool({
    connectionString: configuration.POSTGRES_URL,
  });

  const query = "DELETE FROM users WHERE id = $1";
  pool.query(query, [id], function (err) {
    if (err) return callback(err);

    return callback(null);
  });
}

Note: With Neon, you can manage your database using a few options, such as a Command Line Interface (CLI), the Neon API, or a Structured Query Language (SQL) tool.

After updating the Database Action Scripts, update your Database settings with some sensitive information you used to validate the scripts. For this tutorial, we used the Neon Postgres pooled connection URL.

To get this URL, head back to your Neon dashboard and copy out the Connection String from the connection detail area. Ensure to mark Pooled Connection as true before copying the details.

Back in Auth0's Database settings area, create a key with the name POSTGRES_URL and value with the Pooled Connection string copied from Neon Postgres.

Still within your Auth0 Database Connection, head to Applications and toggle on the Regular Web Applications. We have successfully set Neon Postgres as a custom database for our Auth0 application.

Creating `users` database table on Neon

With the project all but done, there is one tiny thing to add: creating the table for the user store. When creating the Database Action Script, we specified that we are querying a table called users, so let’s make that our table name.

For our table, we will create five fields: id, nickname, email, password, and email_verified. So, head to the SQL Editor in Neon's console and run the query below:

CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE TABLE users(
    id UUID DEFAULT uuid_generate_v4(),
    nickname VARCHAR(255),
    email VARCHAR(255) NOT NULL UNIQUE,
    password VARCHAR(255) NOT NULL,
    email_Verified BOOLEAN DEFAULT FALSE
);

The query above creates a UUID column with a default value generated by uuid_generate_v4() function. It is followed by creating a nickname column — a variable character (varchar) column with a maximum length of 255 characters.

Also, it creates an email and password column, a non-null varchar column with a maximum length of 255 characters, and a unique constraint.

Finally, it creates an email_Verified column, a Boolean column with a default value of false.

When we initialize a signup flow with Auth0, here’s the result on the Neon Postgres database table:

Conclusion

This tutorial taught us how to leverage security and database management by integrating Auth0 with Neon Postgres as a custom database. Pairing Auth0 and Neon Postgres represents a pivotal leap toward fortifying modern applications with scalability, speed, and security. By adopting this strategic combination, developers can ensure a seamless and secure authentication process through Auth0 while leveraging the robust database management capabilities of Neon Postgres.

Resources

Basics of RAG with pgVector and Langchain

Femi-ige Muyiwa — Thu, 08 Feb 2024 08:47:49 +0000

Retrieval augmented generation (RAG) is a technique that enhances the accuracy and reliability of generative AI models by augmenting their knowledge base with facts from external sources. RAG enables large language models (LLMs) to craft accurate, assertive, and brilliant responses on a subject matter.

In this article, we’ll demonstrate how to use the RAG technique in a modern application. To do so, we’ll create a Flutter application using Langchain for the LLM framework and pgVector, an open-source Postgres extension for vector similarity search.

Before beginning, you’ll need a few things:

A good understanding of the Flutter framework and dart programming.
A Neon account.
An OpenAI API key (meaning you’ll need an OpenAI account).
And some cookies and coffee.

Demystifying some concepts

With the aid of databases, especially those that support vector capabilities like Neon, we can use the RAG technique to assist LLMs in delivering accurate answers to an end user. Neon is a fully managed serverless Postgres that provides separate storage and computing to offer autoscaling, branching, and bottomless storage. Neon is fully open source under the Apache 2.0 licenses, and we can find the neondatabase on GitHub.

Let’s first demystify some concepts, starting with pgVector. pgVector is a Postgres extension that works with vector embeddings for storage, similarity search, and more. Enabling the pgVector extension in your Neon database simplifies storing vector embeddings as well as easy querying using the inner product (<#>) or cosine distance (<=>).

Langchain itself is not an LLM but a framework that aids application development with LLMs. Thus, it enables context-aware applications that need language models to reason.

That raises a burning question: How do these parts relate to one another?

RAG applications usually consist of two components: indexing and retrieval.

The indexing process involves integrating (loading) the external data source, splitting it into smaller pieces, embedding the document as a vector, and storing it.

Langchain handles splitting and embedding by providing the application access to OpenAI’s embedding API. Neon comes into play in the storage process.

For the retrieval process, pgVector uses its vector similarity index capability to search the distance between the query vector and the stored vector in the Neon database. Then Langchain uses OpenAI as an LLM to generate the desired result from the query in natural language.

The following sections will cover all the steps in building our application, from creating a Neon database to building the Flutter application. Let us set up a Neon account and create our database without further ado.

Creating a Neon database

After creating a Neon account, as specified earlier, let’s proceed to sign in to the account by selecting one of the methods provided for user authentication.

After successful sign-in, we’ll be redirected to a Create Project screen on the home page, where we are asked to fill in our desired project name, postgres version, and database name. We can explore more options for changing the branch name to any other name, but let’s leave it as main for now and click Create project.

Afterward, we are redirected to the home page, where we get a popup showing the connection details to the Neon project we created earlier. We need these details to access the Neon project from our application and copy it to a safe file. And with that, we have successfully created a Neon database for our Flutter application.

Neon provides three database management methods: the Neon CLI(command line interface), the Neon API, and SQL. With SQL, Neon made an SQL editor available to run SQL commands directly on the console. Thus, we will use SQL to manage our Neon database, but we‘ll do so via a Postgres connection from our application to the Neon database.

The Flutter application is a simple chatbot that responds to queries based on the data from the external data source—in this case, a PDF file. Therefore, in the coming sections, we will clone a Flutter template, connect the template to the Neon database, and add the functionalities to implement the RAG technique within the app.

Creating the Flutter application

To begin, we will use a Flutter template application containing a display area, a text area where we will type our query, and a drawer with a button to upload our desired PDF.

To clone the project, run the command below in a terminal:



git clone https://github.com/muyiwexy/neon_rag_with_langchain.git

After cloning the project, run the following command:



flutter pub get

This command obtains all the dependencies listed in the pubspec.yaml file in the current working directory and their transitive dependencies.

This project uses the Model View Controller (MVC) architecture to handle specific development aspects of the application. The architecture helps us maintain readability by separating the business (core) logic from the UI (presentation layer).

To make things easier to locate, here’s an ASCII representation of the lib folder structure:



lib/
├─ home/
│  ├─ controller/
│  ├─ model/
│  ├─ view/
│  │  ├─ widgets/
│  │  │  ├─ display_area.dart
│  │  │  ├─ text_area.dart
│  │  ├─ home_page.dart
│  ├─ view_model/
├─ core/
│  │  ├─ dependency_injection/
├─ main.dart

Since we are using the MVC architecture, the UI code is placed in the lib/home/view folder. To proceed, we need to add some external dependencies necessary for building the application to the pubspec.yaml file.



dependencies:
  file_picker
  flutter_dotenv
  langchain
  langchain_openai
  path_provider
  postgres
  provider
  syncfusion_flutter_pdf

After successfully doing this, we’ll create an abstraction for all the services needed throughout this project. Let’s call this abstract class LangchainService — within it, we will implement the processes involved in implementing the RAG technique. So, next, locate the lib/home/view_model folder and create a dart file langchain_service.dart within it. To perform an abstraction, add the code below to the file:



abstract class LangchainService {
  // do something
}

Indexing

Load
The load process involves integrating the document into the system, which is usually offline. Thus, to achieve this, we will do the following:

Use the file_picker package to select the files from a local device
Use the syncfusion_flutter_pdf package to read the document (PDF) and convert it to text
Use the path_provider package to find commonly used file ecosystems such as the temp or AppData directories

Compared to the other services, the load process is offline; thus, we will perform this operation separately from the other processes. To load a file, create an index_notifier.dart in the lib/home/controller directory. Next, we create a ChangeNotifier class, IndexNotifier, with a final value of LangchainService. Also, we will create two global private String variables, _filepath and _fileName, and a getter for the _fileName variable.



class IndexNotifier extends ChangeNotifier {
  late LangchainService langchainService;
  IndexNotifier({required this.langchainService});

  String? _filepath;
  String? _fileName;
  String? get fileName => _fileName;
}

In essence, and by the ChangeNotifier, this class will be one of two files that handle the state management load of the application. Next, we will implement a function that returns a type Document from the Langchain package. We will use the method to pick a PDF document from our local device and assign the file type and name to the String variables created earlier.

Also, we will have a Future function that converts PDFs to text, which is loaded as Documents using the TextLoader class from Langchain.




class IndexNotifier extends ChangeNotifier {
  // do something

  Future<Document> _pickedFile() async {
    FilePickerResult? result = await FilePicker.platform
        .pickFiles(type: FileType.custom, allowedExtensions: ['pdf']);
    if (result != null) {
      _filepath = result.files.single.path;
      _fileName = result.files.single.name.replaceAll('.pdf', '').toLowerCase();
      final textfile =
          _filepath!.isNotEmpty ? await _readPDFandConvertToText() : "";
      final loader = TextLoader(textfile);
      final document = await loader.load();
      Document? docs;
      for (var doc in document) {
        docs = doc;
      }
      return docs!;
    } else {
      throw Exception("No file selected");
    }
  }

  Future<String> _readPDFandConvertToText() async {
    File file = File(_filepath!);
    List<int> bytes = await file.readAsBytes();
    final document = PdfDocument(inputBytes: Uint8List.fromList(bytes));
    String text = PdfTextExtractor(document).extractText();
    final localPath = await _localPath;
    File createFile = File('$localPath/output.txt');
    final res = await createFile.writeAsString(text);
    document.dispose();
    return res.path;
  }

  Future<String> get _localPath async {
    final directory = await getApplicationDocumentsDirectory();
    return directory.path;
  }
}

We can load a PDF as a Langchain Document file with the code above.

Split and embed
Now, we need to split and embed the document and store it. To split and embed a Langchain document, we will return to the abstraction created in the langchain_service.dart. There, we will update it with the code below:



abstract class LangchainService {
  List<Document> splitDocToChunks(Document doc);
  Future<List<List<double>>> embedChunks(List<Document> chunks);
}

We will create another file within the same directory called langchain_service_impl.dart to implement this abstraction. Within this file, we’ll implement the LangchainService abstraction created earlier. splitDocToChunks takes in a parameter Document, which is returned from the _pickedFile method in the IndexNotifier class earlier. It then gets the page content.

Then, we use the RecursiveCharacterTextSplitter object to create a document split text into several 1000-character chunks and return it as a Document list.

Next, we will pass the Document list to the embedChunks method, which then creates vector embeddings of this List and returns it as a List< List <double>>.

Below is how the code should look:



class LangchainServicesImpl extends LangchainService {
  final OpenAIEmbeddings embeddings;

  LangchainServicesImpl({
    required this.embeddings,
  });

  @override
  List<Document> splitDocToChunks(Document doc) {
    final text = doc.pageContent;
    const textSplitter = RecursiveCharacterTextSplitter(chunkSize: 1000);
    final chunks = textSplitter.createDocuments([text]);
    return chunks
        .map(
          (e) => Document(
            id: e.id,
            pageContent: e.pageContent.replaceAll(RegExp('/\n/g'), "  "),
            metadata: doc.metadata,
          ),
        )
        .toList();
  }
  @override
  Future<List<List<double>>> embedChunks(List<Document> chunks) async {
    final embedDocs = await embeddings.embedDocuments(chunks);
    return embedDocs;
  }
}

Equally, we will update the IndexNotifier class to control the state of our application while going through all these processes:

index_notifier.dart

Store
So far, we’ve successfully enabled loading, splitting, and embedding the PDF document. Now, we need to store the split and embedded data, which is where the Neon database we created earlier comes in. To do this, we will update the LangchainService abstraction with the code below:



abstract class LangchainService {
  // the abstraction above
  Future<bool> checkExtExist();
  Future<bool> checkTableExist(String tableName);
  Future<String> createNeonVecorExt();
  Future<String> createNeonTable(String tableName);
  Future<String> deleteNeonTableRows(String tableName);
  Future<void> storeDocumentData(Document doc, List<Document> chunks,
      List<List<double>> embeddedDoc, String tableName);
}

The checkExtExist method checks if the vector extension exists and returns the result from the execution. Also, the checkTableExist method checks if a table (the private String variable _filename created earlier) exists within the Neon database and returns the result from the execution, which is a boolean. To do this, we will add the code below to implement the LangchainService in the langchain_service_impl.dart file:

langchain_service_impl.dart

Note: Earlier, we mentioned that Neon allows us to write SQL commands directly on the console through their SQL Editor. Equally, we can execute these SQL commands programmatically from Flutter using the Postgres package.

The methods createNeonVecorExt, createNeonTable, and deleteNeonTableRows, handle the creation of pgVector extension, a Neon database table (the private String variable _filename created earlier), and the deletion of any stored rows (this is in the case the user wants to update the document in the database table and there is a name clash) respectively. When creating the Neon table, we will simultaneously activate vector indexing using the ivfflat algorithm from the pgVector extension. This algorithm provides an efficient solution for approximate nearest neighbor search over high-dimensional data like embeddings.

langchain_service_impl.dart

For the storeDocumentData we will pass the Langchain Document, the chunks, the embedded chunks, and the table name to it and execute an INSERT command in transaction.

langchain_service_impl.dart

Now, we will update the IndexNotifierto implement the changes to our LangchainServices accordingly. We will use the checkExtExist and checkTableExist as conditional checkers to run the createNeonVecorExt, createNeonTable, and deleteNeonTableRows as they satisfy each condition. Here is the updated code below:

index_notifier.dart

We have successfully stored the PDF data within the database table as an id(text), Metadata (Map or JSON), and embedding.

To utilize the ChangeNotifier class within our application, we will mount the ChangeNotifier class using Provider for dependency injection. In this process, we will connect the Neon database and our Flutter application using the Postgres package.

The way to do this is by wrapping the initial stateless widget in the main.dart with a MultiProvider. Doing this mounts our Providers and ChangeNotifierProviders to the widget tree, allowing us to monitor the state of our application easily. Thus, we will head to the lib/core/dependency_injection/ folder, create a file called provider_locator.dart, and paste the code below:

provider_locator.dart

The ProviderLocator class does the following:

Defines a method getProvider that:
- Creates a LangchainService instance.
- Returns a MultiProvider with a LangchainService provider and a ChangeNotifierProvider for IndexNotifier.
Defines a method _createLangchainService that:
- Creates a PostgreSQL connection.
- Creates an OpenAIEmbeddings instance.
- Creates an OpenAI instance.
- Returns a LangchainServicesImpl instance with the created connection, embeddings, and OpenAI.
Defines a method createPostgresConnection that:
- Tries to establish a PostgreSQL connection with specified settings from the Neon connection details earlier.
- If the connection fails, it retries up to a maximum number of times.
- If the connection is not established after maximum retries, it throws an exception.
Defines a method _createEmbeddings that returns an OpenAIEmbeddings instance.
Defines a method _createOpenAIConnection that returns an OpenAI instance.

Note: For security reasons, we will use a .env file to secure our passkey. Kindly follow this article to learn more about how to use flutter_dotenv.

Now, let’s update the main.dart file with the code below:

home.dart

Retrieval

Retrieval is a streamlined process commonly divided into two processes:

Retrieve: This is done by comparing the vector embedding of a user query with the closest available result present in the database. We perform this comparison using the cosine similarity search to compare a vector with another. Thus, when we get the closest results, we can use it for the second process.
Generate: After getting the closest result, we can use it as an assistant for the LLMs to generate responses based on that particular information.

To do this programmatically, we will head to the langchain_service.dart and in the abstraction, add this code below:



abstract class LangchainService {
  // do something
  Future<String> queryNeonTable(String tableName, String query);
}

The method above returns a string response by following the retrieval process above. Here is the code for the implementation below:

langchain_service_impl.dart

The code above does the following:

Implements a method queryNeonTable that:
- Embeds the query using the embeddings object.
- Executes a SQL query on the connection to get similar items from the specified table.
- Converts the result into a list of Metadata objects.
- If Metadata is not empty, it concatenates the page content, creates a StuffDocumentsQAChain object, and calls it with the concatenated content and the original query to get a response.
- If Metadata is empty, it returns a default message: “Couldn’t find anything on that topic”.

We will then create a separate ChangeNotifier class to handle the state of the query. This follows the same pattern as that of the IndexNotifier class with some slight changes. Here is the code below:



import 'package:flutter/material.dart';
import '../view_models/langchain_services.dart';

class Message {
  String? query;
  String? response;
  Message({required this.query, this.response = ""});
}

enum QueryState {
  initial,
  loading,
  loaded,
  error,
}

class QueryNotifier extends ChangeNotifier {
  late LangchainService langchainService;
  QueryNotifier({required this.langchainService});

  final List<Message> _messages = [];

  final _messagesState = ValueNotifier<List<Message>>([]);
  ValueNotifier<List<Message>> get messageState => _messagesState;

  final _queryState = ValueNotifier<QueryState>(QueryState.initial);
  ValueNotifier<QueryState> get queryState => _queryState;

  userqueryResponse(String tableName, String query) async {
    _messages.add(Message(query: query));
    _messagesState.value = List.from(_messages);

    try {
      _queryState.value = QueryState.loading;
      String response = await langchainService.queryNeonTable(tableName, query);
      final List<Message> updatedMessages = List.from(_messages);
      updatedMessages.last.response = response;
      _messagesState.value = updatedMessages;
      _queryState.value = QueryState.loaded;
    } catch (e) {
      // Handle errors if necessary
      print(e);
      _queryState.value = QueryState.error;
      await Future.delayed(const Duration(milliseconds: 2000));
      _queryState.value = QueryState.initial;
    }
  }
}

The code above does the following:

Defines a Message class with query and response fields.
Defines an enum called QueryState with states: initial, loading, loaded, and error.
Creates a QueryNotifier class that extends ChangeNotifier:
- Initializes a LangchainService object.
- Maintains a list of Message objects.
- Defines ValueNotifier objects for messagesState and queryState.
- Defines a method userqueryResponse that:
  - Adds a new Message to _messages.
  - Sets the queryState to loading.
  - Calls queryNeonTable method of langchainService to get a response.
  - Updates the last message’s response and sets queryState to loaded.
  - Handles errors by setting queryState to error, then back to initial after a delay.

After, we will update the getProvider method in the provider_locator.dart file by adding another ChangeNotifierProvider class to the MultiProvider. Here is how the code is below:



class ProviderLocator {
  // provider tree
  static Future<MultiProvider> getProvider(Widget child) async {
    final langchainService = await _createLangchainService();
    return MultiProvider(
      providers: [
        Provider<LangchainService>.value(value: langchainService),
        // IndexNotifier
        ChangeNotifierProvider<IndexNotifier>(
          create: (_) => IndexNotifier(langchainService: langchainService),
        ),
        // QueryNotifier
        ChangeNotifierProvider<QueryNotifier>(
          create: (_) => QueryNotifier(langchainService: langchainService),
        ),
      ],
      child: child,
    );
  }
}

That is it — we should have the result for the application as below:

neon and langchain | Opentape

Muyiwa Femi-Ige - Feb 7th, 3:10pm

app.opentape.io

Here is a link to the repository containing all the code.

Conclusion

Retrieval augmented generation (RAG) enhances LLMs by integrating techniques to ensure a factual and contextual response. The collaboration of a vector database like Neon with the RAG technique and Langchain elevate the capabilities of learnable machines to unprecedented levels. This leads to more brilliant virtual assistants, data analysis tools, and more.

In conclusion, the integration of RAG with pgVector and Langchain is a testament to the incredible prowess of AI and its hopeful future.

Resources

Here are some resources that will guide you more in this journey:

How to add push notifications to your Appwrite Cloud apps

Femi-ige Muyiwa — Thu, 07 Dec 2023 14:38:00 +0000

In today's information-driven landscape, staying current is paramount. Enter the realm of push notifications — an avenue by which data arrives at our digital doorstep, like a knowledgeable neighbor extending a warm invitation. These little alerts have become one of the most essential features in modern-day applications as they update the user about the services that a given application handles.

Let’s explore push notifications a bit further: messages or alerts sent from a server or backend system to a user's mobile device, computer, or other device. These notifications are "pushed" to the device rather than being requested or "pulled" by the user.

In the case of our analogy above, the friendly neighbor is Appwrite. Appwrite is an open-source backend-as-a-service (BaaS) that simplifies repetitive development tasks through simple REST API. Appwrite has added this important feature to its services. In this article, we’ll take a closer look at how we can add push notifications to our Flutter applications using Appwrite's cloud functions.

Before we can proceed, you’ll need to have the following:

An Appwrite cloud account
A Firebase account (subsequently, a Gmail account)
A good foundation in Flutter
And some coffee for the cookies, of course. 😉

Getting started

Appwrite Cloud implements push notifications in its application by combining the Firebase Cloud Messaging (FCM) feature provided by Firebase and its own Appwrite Cloud Function API. Appwrite further simplifies this process programmatically by providing a serverless Node.js code template that allows us to connect our Flutter application to communicate with Firebase Cloud Messaging.

We’re going to get pretty granular in this tutorial, so here’s a helpful outline:

Creating projects:
- Create a Firebase project
- Create an Appwrite project
- Create our Flutter project
Setting up Firebase Cloud messaging:
- Generating a private key
- Creating a Realtime database
Setting up Appwrite:
- Creating an Appwrite database
- Creating an Appwrite Cloud function FCM template
- Customizing the FCM template
Building the Flutter project:
- Cloning the application for the notification device
- Cloning the application for the admin device

Creating projects

Thus, without further ado, let’s create the Firebase project!

Creating a Firebase project

To create a Firebase project, click the earlier link and click Create a project on the page.

Then, Firebase prompts us to fill in a project name, turn on or off analytics (uncheck analytics as it is optional for this project), and finally, continue. That’s it! You have now successfully created a Firebase project.

Creating an Appwrite project

After creating an Appwrite cloud account, log into the console, click Create Project, and fill in a project name and ID here.

Note: ID can also be autogenerated.

Creating a Flutter project

To create a Flutter project, open your terminal and type the command below:

flutter create <project name>

Note: Change the to a name of your choice.

The command above triggers the Flutter SDK to create a new boilerplate project; thus, we have successfully created a Firebase, Appwrite, and Flutter project.

Setting up Firebase Cloud Messaging

To use the Firebase cloud messaging feature within our Appwrite cloud applications, we will need to get the private key and database URL from the Firebase console.

Generating a private key

Starting with the private key, we will navigate to the console's Project settings area by clicking the gear icon.

Next, we will copy the project ID to a safe place, as we will require it when creating the FCM template in Appwrite. We must provide the project ID as a variable, which we will then pass to the FCM template as a parameter.

Next, we'll move to the Service accounts section of the Project Settings area and click Generate new private key, then Generate key. Doing this will download a JSON file containing important information to our PC.

Creating a Realtime database

There are two options for databases to select from within Firebase; for this project, either is effective. For this project, we will choose the Realtime database as our choice. Thus, enlarge the build section within the Firebase console, select Realtime database, and click Create database.

This will prompt a pop-up asking us to set up the database. Select the United States (us-central1) as your preferred database location. Select Start in Test mode for the security rules, as we are not using the database for any production purpose. Then click Enable to create the Realtime database.

Now, the database URL is essential for what we want to build — so let’s copy the URL to a safe place and move to the next item on the list.

Setting up Appwrite

Now that we have successfully gotten the private key and database URL, the next step is to create an Appwrite database and cloud function and customize the FCM cloud function template.

Creating an Appwrite database

After creating an Appwrite project, navigate to the database section within the Appwrite console and click Create database. Next, we will create a collection with a single String attribute called Token. This attribute will be the placeholder for the device token generated from the device we intend to receive the push notification.

After, move to the collection's settings and add permissions by setting the role to any and selecting all the CRUD (create, read, update, and delete) permissions. Setting permissions allows anyone with the database ID and collection ID (in this case, the admin) to access the notification device token.

Creating an Appwrite Cloud function FCM template

Appwrite's application of simple yet effective designs makes navigation relatively smooth — this is evident in the creation of Functions, as all that is required is to select Functions in the console from the navigation bar. To create the FCM template, select the Create function button within the function area.

Then, a pop-up appears, showing all kinds of functions available to the Appwrite cloud. We will proceed to select All templates in the bottom right corner.

In the All templates section, we can filter our template search by use case or runtime or search it by a named keyword. Using the use case filter, we will select the messaging option, select push notification with FCM among the options given, and click Create function.

Appwrite further prompts us to sort out the configuration of the function by setting a name (any name of our choosing) and runtime (Node.js 18.0).

The Variables section is where the primary business lies, where we must add some information in the private key JSON file and the database URL copied from the Firebase project earlier. So, from the private key JSON file, we will copy out the project ID (alternatively, we can get the project ID from the Firebase project settings), private key, and client_email. Also, we will add the URL from the Realtime database — then, we are good to go.

In the Connect section, we can add the template to an available repository or create a new one. For the tutorial's sake, we will select Create a new repository, then click next to set up the Git organization and the repository name.

Finally, we can specify our preferred Git branch intended for the template (it is main by default). After completing all the steps, we can create a push notification FCM template.

Customzing the FCM template code

To customize the FCM template code, head to the repository containing the template code on GitHub, and from the code section, navigate to the src folder. The src folder contains two files: main.js and utils.js.

Update the main.js file with the code below:

import { throwIfMissing, sendPushNotification } from './utils.js';

export default async ({ req, res, log, error }) => {
    throwIfMissing(process.env, [
        'FCM_PROJECT_ID',
        'FCM_PRIVATE_KEY',
        'FCM_CLIENT_EMAIL',
        'FCM_DATABASE_URL',
    ]);

    try {
        throwIfMissing(req.body, ['deviceToken', 'message']);
        throwIfMissing(req.body.message, ['title', 'body']);
    } catch (err) {
        return res.json({ ok: false, error: err.message }, 400);
    }

    log(`Sending message to device: ${process.env.FCM_PROJECT_ID}`);

    try {
        const response = await sendPushNotification({
            notification: {
                title: req.body.message.title,
                body: req.body.message.body,
            },
            token: req.body.deviceToken,
        });
        log(`Successfully sent message: ${response}`);

        return res.json({ ok: true, messageId: response });
    } catch (e) {
        error(e);
        return res.json({ ok: false, error: `Failed to send the message ${e}` }, 500);
    }
};

Also, we will update the utils.js file with the code below:

import admin from 'firebase-admin';

/**
 * Throws an error if any of the keys are missing from the object
 * @param {*} obj
 * @param {string[]} keys
 * @throws {Error}
 */
export function throwIfMissing(obj, keys) {
    const missing = [];
    for (let key of keys) {
        if (!(key in obj) || !obj[key]) {
            missing.push(key);
        }
    }
    if (missing.length > 0) {
        throw new Error(`Missing required fields: ${missing.join(', ')}`);
    }
}

/**
 * @param {admin.messaging.Message} payload
 * @returns {Promise<string>}
 */

export async function sendPushNotification(payload) {
    if (!admin.apps.length) {
        admin.initializeApp({
            credential: admin.credential.cert({
                projectId: process.env.FCM_PROJECT_ID,
                clientEmail: process.env.FCM_CLIENT_EMAIL,
                privateKey: process.env.FCM_PRIVATE_KEY.replace(/\\n/gm, "\n"),
            }),
            databaseURL: process.env.FCM_DATABASE_URL,
        });
    }
    return await admin.messaging().send(payload);
}

After doing that, let’s commit our changes. Due to Appwrite’s connection to our GitHub, the commit automatically triggers the cloud Function to redeploy as it monitors changes in the attached repository.

Now, we’re ready to clone the Flutter applications — both the admin (trigger application) and push notification application (receiver application).

Building the Flutter project

Earlier in the article, we indicated that we would build two Flutter applications during the project. One is the device receiving the push notification; the other will be an admin device that triggers the Appwrite cloud FCM template. Doing this gives us a better use case for push notifications programmatically. Let’s get into it.

Cloning the application for the notification device

In this section, we will clone the application for the notification device from this repository. The application’s UI contains a button that uploads the phone token obtained from an FCM request.

After cloning the application to local storage, open it in your preferred code editor and run the command below:

flutter pub get

This command obtains all the dependencies listed in the pubspec.yaml file in the current working directory and their transitive dependencies. Next, run the command flutter run, and the application should look like the video below:

To send a push notification, a token from the recipient device must be available to the Firebase cloud messaging service. The question now is, how can we achieve that?

The first step involves adding our Flutter project to the Firebase project created earlier. So, starting with that, we can achieve that manually through the Firebase console or using the Firebase CLI (command line interface).

The Firebase CLI is a simple tool that allows easy interactions with Firebase using the command line. The Firebase CLI provides commands that let us send specific requests to Firebase and get adequate responses. Some of these requests include the following:

Project Setup
Authentication and Authorization
Real-time Database
Hosting
Cloud Functions
Cloud Storage
Firestore
Test Lab
Analytics

Thus, to use the Firebase CLI for project setup within a Flutter project, let’s head back to the Firebase console. Within the base console are two options for connecting an Android application to Firebase. So, we will select the second option that says Flutter.

When we click the option, the first step asks that we install the Firebase CLI. According to the link, there are several methods we can follow to install the Firebase CLI, depending on our device’s operating system. We will use the node package manager (npm) to install the CLI for uniformity. Here is a guide on how to install Node.js on your device.

After installing Node.js, run the command below within your terminal:

npm install -g firebase-tools

Doing this automatically installs the Firebase CLI to our preferred device, and with that, we can proceed to the next step of logging into our Firebase console via the CLI. To do that, we will use the command:

firebase login

After logging into the Firebase CLI, we must install the FlutterFire plugin to create a new Firebase project in Dart. FlutterFire is a set of plugins that connects our Flutter project to Firebase. To activate the FlutterFire plugin on our PC, use the following command:

dart pub global activate flutterfire_cli

To use the FlutterFire plugin to configure our Firebase project within the Flutter app, we must open a terminal within the Flutter project folder and then run this command:

flutterfire configure --project=<Firebase Project ID>

Note: Remember to change the <Firebase Project ID> in the command above to the ID of the Firebase project.

After selecting what platform(s) the configuration should support, the CLI will add some essential files to the Flutter project, and we are good to proceed.

Next, we will connect our Flutter application to Appwrite. We do this to upload the generated token from our notification device to a general database — this allows a central admin to access the token and send push notifications accordingly to several devices.

Here is how to connect an IOS or Android device in Flutter to Appwrite:

iOS

First, obtain the bundle ID by navigating to the project.pbxproj file (ios > Runner.xcodeproj > project.pbxproj) and searching for the PRODUCT_BUNDLE_IDENTIFIER.

Next, navigate to the Runner.xcworkspace folder in the application’s iOS folder in the project directory on Xcode. To select the runner target, choose the Runner project in the Xcode project navigator and find the Runner target. Next, select General and IOS 11.0 as the target in the deployment info section.

Android

For Android, copy the XML script below and paste it below the activity tag in the Androidmanifest.xml file (to find this file, head to android > app > src > main).

Note: change [PROJECT-ID] to the ID you used when creating the Appwrite project.

We will also need to set up a platform within the Appwrite console. Follow the steps below to do so.

Within the Appwrite console, select Create Platform and choose Flutter for the platform type.
Specify the operating system: in this case, Android.
Finally, provide the application and package names (found in the app-level build.gradle file).

Cloning the application for the admin device

In this section, we will clone the application for the admin device. The application’s UI contains a button that triggers the cloud function. Let’s clone the repository specified in the prerequisites.

After cloning the application to local storage, we will open it in our preferred code editor and run the command below:

flutter pub get

So, when we click the Send Push Notification button, it triggers this function in the app_controller.dart file:

void sendNotification(String token) async {
    try {
      final Map < String, dynamic > requestBody = {
            'deviceToken': token,
                'message': {
                'title': 'Notification Title',
                    'body': 'Notification Body',
        },
        };
      final result = await functions.createExecution(
            functionId: Appconstants.functionId,
            body: jsonEncode(requestBody),
            path: '/',
            method: 'POST',
            headers: {
            'Content-Type': 'application/json',
        },
        );
    } catch (e) {
        print(e);
    }
}

This function encodes a Map<String, dynamic> into a JSON and programmatically sends a POST request to the Appwrite Function. Without an admin page, we can trigger these notifications within the Appwrite console.

To do this, navigate to the Function section of the console, and within the FCM cloud function template, select Execute now.

We will leave the Path as it is in the pop-up, but we‘ll change the method to POST. In the Advanced settings section within the pop-up, add a Header with its name being Content-Type and value being application/json. Finally, in the body section, we will add a JSON that looks like the format below:

{
    "deviceToken": "<notification device token>",
        "message": {
        "title": "hi",
            "body": "how are you"
    }
}

Then we will execute the function, and the result should look like the GIF below:

Conclusion

Push notifications are hugely important in modern applications. By incorporating push notifications into our Appwrite Cloud applications, we can keep our users informed and engaged. Leveraging this powerful communication channel ensures that our audience receives timely updates, personalized messages, and essential information — enhancing their overall experience.

With our backend configuration and error handling in place, we can reliably deliver push notifications while minimizing potential issues. Integrating push notifications into our applications is a valuable strategy for boosting user engagement and ensuring effective communication with our user base.

Resources

Here are some additional resources that you might find useful.

Implementing OAuth2 Clients with Flutter made simple

Femi-ige Muyiwa — Thu, 07 Sep 2023 11:47:38 +0000

Having to input our information every time we register for a new service is tiring. With open authorization or OAuth2, we no longer have to undergo that pain. Open authorization is a design that lets websites and applications access certain hosted resources from a provider by request from the user. These resources can range from the user’s registered email address to personal information to cloud data.

This article will show how to implement OAuth2 in a Flutter application with Google as the provider and Appwrite as the OAuth2 Client.
Here is the link to the repository containing the code for this project.

Prerequisites

The following are required to follow along with this tutorial:

Basic understanding of Dart and Flutter
Flutter SDK installed
Xcode with a developer account
Appwrite account; s ign-up is completely free
A Gmail account; s ign-up is completely free

Creating an Appwrite project

After registering on the Appwrite cloud platform, we will be redirected to an onboarding page where we will have to create a new project. We will fill in our preferred project name and ID (optional as ID can be autogenerated), then select Create Project.

After creating a new project, we must register our application to a specific platform within our project. Therefore, in the Add platform area of the Home section, we will select the Flutter App button and select our preferred operating system (in this case, Android). Then, we will fill in the application and package names (the package name is in the app-level build.gradle file).

Getting started

We will start by cloning the project repository to our PC. This repository contains the template and code we will use for the project. To clone the repository, run the command below:

git clone https://github.com/muyiwexy/google_auth_flutter.git

After the cloning process, we will run the command flutter pub get to load all the packages used in the project to our PC.

Finally, we will run the project using the command flutter run and we should have the result below:

To use our Appwrite instance within our Flutter app, we need to connect our project to Appwrite. This process differs from the emulator's OS (Android or IOS).

iOS
First, obtain the bundle ID by going to the project.pbxproj file (ios > Runner.xcodeproj > project.pbxproj) and searching for the PRODUCT_BUNDLE_IDENTIFIER.

Now, head to the Runner.xcworkspace folder in the applications iOS folder in the project directory on Xcode. To select the runner target, choose the Runner project in the Xcode project navigator and find the Runner target. Then, select General and IOS 11.0 in the deployment info section as the target.

Android
For Android, copy the XML script below and paste it below the activity tag in the Androidmanifest.xml file (to find this file, head to android > app > src > main).

Note: change [PROJECT-ID] to the ID used when creating the Appwrite project.

Enabling Google OAuth2 provider
In order to use Google as a provider in Appwrite, we will need to have it enabled. To do this, we will head to Auth > Settings. We will scroll to Google and check the enabled switch within the settings area. We are also required to fill in the App ID and secret, but we will do that later as we initially need to set up Google OAuth.

Setting up Google OAuth 2.0
To set up, we will head to the Google API console page and head to API and Services within our project (we must create a new project if we do not have one).

Next, we select Credentials, followed by clicking on CREATE CREDENTIALS. Doing this will show a dropdown with some options, and we will proceed to select OAuth client ID.

Note: If it is your first time creating a client ID, you will need to configure your consent screen, which you won’t be required to do again after.

When creating a new OAuth Client ID, we will follow the configurations below:

Application Type - Web Application
Name - (choose preferred name)
Authorized JavaScript origins
- URIs1 - https://cloud.appwrite.io
- URIs2 - https://localhost
Authorized redirect URIs
- URIs - paste URL from the Google OAuth provider provided by Appwrite

After filling in the prompts on the page, we select save. Then we will get a splash screen showing our client ID and secret.

We will then head back to the Google OAuth provider page, paste in the Client ID and secret in the App ID and secret section, respectively, and click update. That is all, and we have successfully enabled Appwrite as our OAuth2 client.

Google OAuth2 provider and Flutter
This project utilizes the provider package for its state management. In the util.dart file, we will create an Appconstants class to serve as placeholders for some important constants, such as our Appwrite endpoint and ProjectID. Next, create a ChangeNotifier class called AppProvider. This class initializes the client and account instance from Appwrite. Next, create a function called socialSignIn which takes in the OAuth2 provider as a string and context. This function will use the Account API to create a new OAuth2 session on Appwrite and run the condition that if an account is successfully created, it redirects to a new page by setting _isLoggedin to true.

import 'package:appwrite/appwrite.dart';
import 'package:flutter/material.dart';
import 'package:google_auth/screens/home.dart';

class Appconstants {
  static const String endpoint = "<endpoint or hostname>";
  static const String projectid = "<project ID>";
}

class AppProvider extends ChangeNotifier {
  Client client = Client();
  late Account account;
  late bool _isLoggedIn;
  bool get isLoggedIn => _isLoggedIn;
  AppProvider() {
    _isLoggedIn = true;
    initialize();
  }
  initialize() {
    client
      ..setProject(Appconstants.projectid)
      ..setEndpoint(Appconstants.endpoint);
    account = Account(client);
    checkLogin();
  }
  checkLogin() async {
    try {
      await account.get();
    } catch (_) {
      _isLoggedIn = false;
      notifyListeners();
    }
  }
  socialSignIn(String provider, context) async {
    await account
        .createOAuth2Session(
      provider: provider,
      success: "",
      failure: "",
    )
        .then((response) {
      _isLoggedIn = true;
      notifyListeners();
    }).catchError((e) {
      _isLoggedIn = false;
      notifyListeners();
    });
  }
}

To use this code from our ChangeNotifier class, we will create a UI consisting of a button and an alternate screen called Homepage, which we will route to after successfully running the socialSignIn function. Therefore, in our lib folder, create an alternate folder called screens and create two files in it, user_reg.dart and home.dart.

The user_reg.dart screen’s UI will contain an elevated button in which its onPressed property triggers the socialSignIn function in the ChangeNotifier class.

import 'package:flutter/material.dart';
import 'package:google_auth/util.dart';
import 'package:provider/provider.dart';
class UserRegistration extends StatefulWidget {
  const UserRegistration({super.key, required this.title});
  final String title;
  @override
  State<UserRegistration> createState() => _UserRegistrationState();
}
class _UserRegistrationState extends State<UserRegistration> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: IntrinsicWidth(
          child: Container(
            padding: const EdgeInsets.all(20.0),
            decoration: const BoxDecoration(
              color: Colors.blueGrey,
              borderRadius: BorderRadius.all(Radius.circular(20.0)),
            ),
            child: ElevatedButton.icon(
              onPressed: () {
                final state = Provider.of<AppProvider>(context, listen: false);
                state.socialSignIn("google", context);
              },
              icon: const Icon(Icons.g_mobiledata),
              label: const Text("Sign in with Google"),
            ),
          ),
        ),
      ),
    );
  }
}

For the home.dart file, we will have a StatefulWidget called Homepage consisting of text at the center of the page.

import 'package:flutter/material.dart';
class Homepage extends StatefulWidget {
  const Homepage({super.key});
  @override
  State<Homepage> createState() => _HomepageState();
}
class _HomepageState extends State<Homepage> {
  @override
  Widget build(BuildContext context) {
    return const Scaffold(
      body: Center(
        child: Text("data"),
      ),
    );
  }
}

When we run the app, we should have our results as below:

Conclusion

This post showed how to use Google as a provider using Appwrite as an OAuth client. Appwrite utilizes Google as a provider and serves as a client for 35 more providers. Thus, creating a vast plain of knowledge to explore.

Here are some resources to aid the learning journey:

SMS authentication using Appwrite and Twilio

Femi-ige Muyiwa — Tue, 05 Sep 2023 22:36:47 +0000

When checking out an app or service, it is exhilarating, but it can also be deflating when asked to sign up to access significant parts of that service. We can attribute this sudden change in emotions to the tedious steps associated with the user sign-up process.

Though tedious, most sign-up procedures are necessary as they ensure the safety of the user's personal information. Let's face it; no one likes knowing their personal information is easily accessible to others, right? But come to think of it, what if there is an option where we don't have to provide much information while maintaining user information security? We like that option much better!

With Appwrite, developers can integrate several authentication features into their applications with its diverse APIs and features. One feature that catches the eye is phone authentication, which uses Appwrite's Account API to create user sessions with phone numbers. This feature's authentication flow involves sending a secret code (one-time password) via an SMS provider such as Twilio, TextMagic, Telesign, MSG91, or Vonage. This article will use Twilio as our primary SMS provider and show how to set it up in Appwrite.

This article requires the reader to have a Twilio account, iOS Simulator, Android Studio, or Chrome web browser to test their application, and an Appwrite instance running on Docker, DigitalOcean droplet, or Gitpod.

Note: Check out this article to learn how to set up an Appwrite instance.

Setting up Twilio in Appwrite

Getting Twilio account information
The purpose of creating a Twilio account is to allow us to have three things:

Account SID
Auth Token
Twilio Phone Number

After logging in, we will need to follow the welcome steps, which guide us on how to get a custom Twilio number. In the first step, we will click on Get Twilio Phone Number, then scroll down to copy the information.

After copying the account information, we must update Appwrite's .env file with the data.

Updating Appwrite's environmental variables
We will head to our PC's Appwrite directory and update the .env file by giving the _APP_SMS_PROVIDER and _APP_SMS_FROM variables the values sms://[ACCOUNT SID]:[AUTH TOKEN]@twilio and [TWILIO PHONE NUMBER] respectively.

After that, it is important to restart the Appwrite server after updating the .env file. This allows us to use the environmental variables added to the file, and we can do this using the command docker compose up -d --force-recreate in a terminal. The command recreates every image within the Appwrite container running on docker.

Note: The docker service must be running, and make sure to run the command terminal within the directory of the .env file.

We are now ready to utilize Twilio to create our phone authentication. Thus, in the next section, we will create a Flutter application that routes to the homepage after authenticating a user's phone number. But first, we need to create an Appwrite project; Thus, head to the browser and enter the IP address or hostname specified when creating the Appwrite instance. Next, we must create a new project by clicking Create Project and filling in the project name and ID (optional).

Creating a Flutter project
In this article, we will use a Flutter template consisting of a login page with input fields, a submit button, and a homepage to route to after authentication. Therefore, run the command below in a terminal to clone the Flutter project from GitHub.

git clone https://github.com/muyiwexy/main_twilio_auths.git

Note: Check out the official GitHub docs to learn about cloning a repository.

After cloning the UI, we must obtain all the dependencies in the pubspec.yaml file, or there will be "some" errors. So, we can do that with the command flutter pub get.

The project already has the appwrite: ^9.0.1 dependency, though, for a new project, we need to head to the pubspec.yaml file and add the appwrite: ^9.0.1 to the dependencies section. Then run the flutter pub get command or save the file, and the command runs automatically (depending on the code editor).

To run the project, we can use the command:

Flutter run

After that, our template should look like the image below:

This template is a husk, but it will come to life in the next section as we work on adding some functionality to the application and connecting it to Appwrite.

Creating phone authentication logic
In the appwrite_auth.dart file, we will create a class AppwriteAuth, and within it, we need to initialize the Client instance from Appwrite.

import 'package:appwrite/appwrite.dart';

class AppwriteAuth {
  final Client _client;
  AppwriteAuth()
      : _client = Client()
          ..setEndpoint("[endpoint or hostname]")
          ..setProject("[projectID]");
}

Next, we will create a Future function of type Token called phoneLogin. The function uses Appwrite's account API to create a session using a user's phone number by providing a phone number and ID.

import 'package:appwrite/appwrite.dart';
import 'package:appwrite/models.dart' as models;

class AppwriteAuth {
  final Client _client;
  AppwriteAuth()
      : _client = Client()
          ..setEndpoint("[endpoint or hostname]")
          ..setProject("[projectID]");

  Future<models.Token> phoneLogin(String phoneNumber) async {
    final account = Account(_client);
    try {
      final response = await account.createPhoneSession(
        userId: ID.unique(),
        phone: phoneNumber,
      );
      return response;
    } catch (e) {
      rethrow;
    }
  }
}

The function above only creates a user session, which triggers sending the secret token but doesn't verify the user. To verify the user, we will create another Future function of type Session called verifyPhoneOTP. The Future also utilizes Appwrite's account API to update a session rather than create one. This means we must provide the user's ID to check whether such a user exists and the secret token sent to verify the user. This completes our authentication flow.

import 'package:appwrite/appwrite.dart';
import 'package:appwrite/models.dart' as models;

class AppwriteAuth {
  final Client _client;
  AppwriteAuth()
      : _client = Client()
          ..setEndpoint("[endpoint or hostname]")
          ..setProject("[projectID]");

  Future<models.Token> phoneLogin(String phoneNumber) async {
    // do something
  }

  Future<models.Session> verifyPhoneOTP({
    required String userID,
    required String otp,
  }) async {
    final account = Account(_client);
    try {
      final session = await account.updatePhoneSession(
        userId: userID,
        secret: otp,
      );
      return session;
    } catch (e) {
      rethrow;
    }
  }
}

To use this function, head to the user_registraton.dart file and update the _createUserSession with the code below:

    import 'package:appwrite/models.dart';
    import 'package:flutter/material.dart';
    import 'package:main_twilio_auth/auth/appwrite_auth.dart';
    import 'package:main_twilio_auth/pages/home.dart';
    import 'package:main_twilio_auth/widgets/button.dart';
    import 'package:main_twilio_auth/widgets/input_form_field.dart';

    class UserLogIn extends StatefulWidget {
      const UserLogIn({Key? key}) : super(key: key);
      @override
      State<UserLogIn> createState() => _UserLogInState();
    }
    class _UserLogInState extends State<UserLogIn> {
      final TextEditingController _phoneController = TextEditingController();
      final TextEditingController _otpTokenController = TextEditingController();
      final GlobalKey<FormState> _formKey = GlobalKey<FormState>();
      bool _isVisible = false;
      bool _isButtonDisabled = true;
      Token? _response;

      void _createUserSession() async {
        if (_formKey.currentState!.validate()) {
          _response = await AppwriteAuth().phoneLogin(_phoneController.text);
          setState(() {
            _isVisible = true;
          });
          await Future.delayed(const Duration(seconds: 5));
          setState(() {
            _isButtonDisabled = !_isButtonDisabled;
          });
        }
      }
       @override
      Widget build(BuildContext context) {
        return Placeholder();
      }
    }

In the code above, we added a Token instance _response from the Appwrite's models package. This allows us to map the JSON response received from its associated value to a model class, and that's shown when we call the phoneLogin function in the code above.

Next, we will update the _updateUserSession with the code below:

import 'package:appwrite/models.dart';
import 'package:flutter/material.dart';
import 'package:main_twilio_auth/auth/appwrite_auth.dart';
import 'package:main_twilio_auth/pages/home.dart';
import 'package:main_twilio_auth/widgets/button.dart';
import 'package:main_twilio_auth/widgets/input_form_field.dart';

class UserLogIn extends StatefulWidget {
  const UserLogIn({Key? key}) : super(key: key);
  @override
  State<UserLogIn> createState() => _UserLogInState();
}
class _UserLogInState extends State<UserLogIn> {
  final TextEditingController _phoneController = TextEditingController();
  final TextEditingController _otpTokenController = TextEditingController();
  final GlobalKey<FormState> _formKey = GlobalKey<FormState>();
  bool _isVisible = false;
  bool _isButtonDisabled = true;
  Token? _response;

  void _createUserSession() async {
    // do something
  }

  void _updateUserSession() async {
    if (_formKey.currentState!.validate()) {
      await AppwriteAuth().verifyPhoneOTP(
          userID: _response!.userId, otp: _otpTokenController.text);
      if (!mounted) return;
      await Navigator.of(context).pushReplacement(MaterialPageRoute(
        builder: (context) => const HomePage(),
      ));
      _phoneController.clear();
      _otpTokenController.clear();
    }
  }
   @override
  Widget build(BuildContext context) {
    return Placeholder();
  }
}

This function calls the verifyPhoneOTP from the AppwriteAuth class, checks if the current widget is still mounted, then routes by replacing the UserLogIn widget with the HomePage widget on the widget tree.

When we don't get the secret token due to some factors, the "send otp" button triggers a function reSendOtp which recalls the phoneLogin function.

void _sendOtp() async {
  setState(() {
    _isButtonDisabled = true;
  });
  await AppwriteAuth().phoneLogin(_phoneController.text);
  Future.delayed(const Duration(seconds: 5), () {
    setState(() {
      _isButtonDisabled = !_isButtonDisabled;
    });
  });
}

Creating an Appwrite platform and connecting Flutter to Appwrite
After creating an Appwrite project, we will register our application by creating a new Flutter platform within the Appwrite instance. So, we will click Flutter App, select our desired operating system (in this case, Android), and fill in the Name and Package Name.

The app's package name can be found in the app-level build.gradle file

iOS
First, obtain the bundle ID by navigating to the project.pbxproj file (ios > Runner.xcodeproj > project.pbxproj) and searching for the PRODUCT_BUNDLE_IDENTIFIER.

Next, head to the Runner.xcworkspace folder in the application's iOS folder in the project directory on Xcode. To select the runner target, choose the Runner project in the Xcode project navigator and find the Runner target. Next, select General and IOS 11.0 as the targeting of the deployment info section.

Android
For Android, copy the XML script below and paste it below the activity tag in the Androidmanifest.xml file (to find this file, head to android > app > src > main).

Note: change [PROJECT-ID] to the ID you used when creating the Appwrite project.

Finally, running the application should have our result like the gif below:

Conclusion

This tutorial has shown how to create a phone authentication in Flutter using Appwrite as the backend service and Twilio as an SMS provider. Authentication is a focal point in app development, and innovations to simplify the process are always happening. Appwrite has simplified this process by cutting the extra work of creating an SMTP flow or token, which is just cumbersome to a simple copy-and-paste job while ensuring user data security data.

Resources

Build fast with Appwrite Cloud: accelerate your web, mobile, and Flutter development

Femi-ige Muyiwa — Wed, 12 Jul 2023 13:33:03 +0000

Speed is crucial for prototyping, building, and deploying modern-day web and mobile applications. This necessity has led to a vast increase in third-party tools providing various services like authentication, storage, and emails which have all helped developers scale more quickly. While these tools have effectively delivered faster results, implementing, maintaining, and tracking multiple third-party tools in an application can be challenging and time-consuming, leaving room for errors. With speed as the aim, having a service that combines the capacity of a backend and third-party tools will come in handy.

This article introduces Appwrite cloud, which offers a nearly unlimited amount of possibilities for scaling and accelerates the development process of your mobile, web, and Flutter applications.

Appwrite Cloud features

Appwrite Cloud takes being capable to a whole new level by providing backend functionalities to your mobile, web, and Flutter applications. Here are some of the capabilities of Appwrite Cloud:

User Authentication and Integration with third-party services
Database Management
File Storage and Management
Real-time Communication

User authentication and integration with third-party services
Appwrite Cloud provides easy and scalable ways to implement secure user authentication, login, and registration in your applications. User data is kept safe through Appwrite's effort to employ a safety mechanism that protects data during transmission and when at rest. Appwrite Cloud's compliance with regulations like GDPR (General Data Protection Regulation) ensures that your applications meet a required data privacy standard.

Database management
Database management is essential to a modern-day backend and Appwrite Cloud accomplishes it like no other. It allows you to perform the basic CRUD (create, read, update, and delete) operations of a traditional database and other advanced database operations by providing a NoSQL document-based database. It offers flexibility and scalability when storing and managing structured data, thus, enabling efficient data management and querying for your applications.

File storage and management
File storage, versioning, control, and generation have become efficient using Appwrite Cloud. Appwrite Cloud includes a scalable file storage system that allows you to upload, store, and manage images, videos, and document files.

Real-time communication
Appwrite Cloud allows your activities to come to life in real-time. "Real-time" is a feature that uses WebSocket, allowing instant sharing and data synchronization across multiple clients. Consider it: collaborative features like live chats, updates, and editing all come to life using Appwrite Cloud.

Benefits of using Appwrite Cloud for fast development

When it comes to assisting developers in rapid iterating and prototyping, Appwrite Cloud offers several benefits, like the provision of prebuilt components and SDKs, scalability and performance optimization, and simple hosting and deployment.

Prebuilt components and SDKs
The features mentioned earlier assist in streamlining development by providing prebuilt features to the developers. For example, developers can quickly integrate user authentication into their prototypes using Appwrite Cloud Account API rather than creating user authentication from scratch.

Scalability and performance optimization
Appwrite Cloud also eliminates scalability and performance-related concerns by being designed to upscale simultaneously with the application's growth. Therefore, you can rely on Appwrite's infrastructure to ensure your prototype performs as expected, thus, allowing you to focus better on the user experience side of things during the iteration stage.

Simple hosting and deployment
Appwrite Cloud provides seamless deployment and hosting options while eliminating the need to manage servers and infrastructures. This option enables rapid testing and iteration, thus, making it easy and quick for you to share your prototype with clients for feedback.

Why is the need for eliminating server and infrastructure management a big deal?
Imagine you must build a house (your application) and need a piece of land (server infrastructure) to achieve the goal. Typically, you must find and prepare the land (plumbing, stump removal, and secondary electricity), which can be quite tasking.
In this case, Appwrite Cloud is like ready-made ground; you only have to build on it. The level of effectiveness doesn't just make Appwrite Cloud time-efficient but borderline cost-effective.

Boosting web and app development with Appwrite Cloud

Appwrite Cloud doesn't only boost development time but also lets you expand faster by providing the following capabilities:

Cross-platform compatibility
Effortless data synchronization
Integration with popular web and app frameworks
Push notifications and in-app messaging

Cross-platform compatibility
Appwrite Cloud supports cross-platform development, allowing developers to build web and mobile applications (IOS and Android) using the same codebase. This advantage saves time and boosts developers' efficiency by enabling them to write and deploy code quickly across different platforms.

Effortless data synchronization
Appwrite Cloud's "real-time" feature lets you easily synchronize data between your mobile, web applications, and the backend server. Changes made on one device are automatically reflected on other devices, allowing users to have up-to-date information regardless of their device.

Integration with popular web and app frameworks
Appwrite Cloud integrates with several web and mobile frameworks for development, such as Flutter. Flutter is a cross-platform UI toolkit that lets developers build web, mobile, and desktop applications using a single codebase. Appwrite Cloud provides backend integration for this framework by providing ready-to-use SDKs and components, accelerating development.

Push notifications and in-app messaging
Through server-side functions, you can add a custom function to perform a specific operation once a user is within the application. Such operations are push notifications and in-app messaging. These sorts of functions facilitate personalized interactions and improve the user's overall experience.

Improving Flutter development with Appwrite Cloud

As mentioned earlier, Flutter is a cross-platform UI toolkit for web, mobile, and desktop applications. Flutter also provides a native-like performance, thus, giving it this silky smooth experience. Appwrite Cloud seamlessly integrates with a Flutter application's interface, enabling a fast response between the client and the backend. This interaction extends to simple features like the hot reload, which instantly allows developers to see the changes made in their code without rebuilding the entire app.

Appwrite Cloud provides Flutter applications with secure authentication and data storage. It offers robust encryption and data protection mechanism using secure communication protocols, such as HTTPS (HTTP over SSL/TLS). This communication protocol ensures that data transmitted between your app and the backend servers is encrypted.

Getting started with Appwrite Cloud

To use Appwrite Cloud for your mobile, web, and Flutter development, follow the steps below:

Start by creating a new account on the Appwrite Cloud platform. You can register with your email and password or via a GitHub account.
After signing up, you will get redirected to the onboarding page to create a new project. Fill in the project name and ID, then select Create Project. Congrats, you have successfully created an Appwrite Cloud account.

Appwrite also provides the documentation needed to get you started with their APIs and components. Head to the docs section of their website to get started.

According to Appwrite's latest community post by Christy Jacob, they are currently working on a pricing model to benefit small and large organizations and independent programmers. They also announced they would include a free tier to appeal to hobby project developers. So, sit tight and get ready to create.

Conclusion

In summary, Appwrite Cloud epitomizes effectiveness, speed, and cost efficiency. It provides developers with the necessary tools to scale up their web, mobile, and Flutter applications by providing components of a traditional backend and third-party tools all in one package.

Resources

How to build a functional food delivery tracker app with Appwrite relationship database and Flutter

Femi-ige Muyiwa — Wed, 14 Jun 2023 14:12:25 +0000

Food delivery is one of the most prominent businesses in the digital age. Companies like Uber Eats, Jumia, Mano, etc. have applications to track customers’ orders, and in turn, the customer can view the delivery process of their products. These features can be enabled by establishing a Relationship in databases, thus creating relationships between order and delivery.

With Appwrite, you can now use such features in your project. Thus, this article will demonstrate how to create a functional delivery tracker application using the Appwrite relationship database and Flutter.

What we’ll cover:
A food delivery tracker application should offer the following features:

Create user’s order
View the user’s order
Confirm the order’s delivery
Cancel the user’s order and delivery.

Prerequisites

This tutorial requires the reader to meet the following requirements:

Xcode (with developer account for Mac users).
iOS Simulator, Android Studio, or Chrome web browser to run the application.
An Appwrite instance running on either Docker, DigitalOcean droplet, or Gitpod. Check out this article for the setup.

Setting up the Appwrite project

Appwrite’s relationship database feature is currently only available locally; thus, you must have an Appwrite local instance running on Docker or DigitalOcean droplet. Once you have this, create a project and fill in the project's name and ID.

You will need to create multiple database collection pairs during this project. The first pair will be between vendors → foodtype, and the second pair order → delivery.

Head to Databases, select Create Database, and fill in the database name and ID. After, create a new collection by selecting Create collection and filling in the subsequent name and ID.

Next, you need to add some attributes as they will serve as parameters to hold data within your collection. To add an attribute to a particular collection, head to any of your newly created collections, select Create attribute and select any of your preferred attribute types from the available options.

Therefore, add the attributes below to the vendors collection:

key	Type
name	String
foodTypes	Relationship with foodTypes

The foodtype collection will have the following attributes:

key	type
foodtype	String
food	String []
foodChecked	Boolean []

The relationship attribute between the vendors → foodtype has the following settings:

One-way relationship
Related Collection → vendor
Attribute Key → vendors
Attribute Key (related collection) → foodTypes
Relation → many to many
On deleting a document → set Null

For the second pair, the order collection has the following attributes:

key	Type	Default value
vendorName	String	-
orderStatus	Enum(in-transit, payment-acepted, packed, delivered, canceled, not-delivered)	payment-acepted
orderTime	String	-
foodItems	String []	-
delivery	Relationship with delivery	-

The delivery collection has the following attributes:

key	Type	Default value
OrderID	String	-
deliveryStatus	Enum (in-progress, completed, failed)	in-progress
deliveryTime	String	-
order	Relationship with order	-

The relationship attribute between the order → delivery has the following settings:

Two-way relationship
Related Collection → delivery
Attribute Key → delivery
Attribute Key (related collection) → order
Relation → one to one
On deleting a document → cascading

Finally, head to the collection settings section and set the collection level permission to role:any and check the read CRUD permissions vendors → foodtype pair. For the order → delivery pair, set the collection level permission to role:user and check all the CRUD permissions

These permissions allow anyone to create, read, update, and delete documents from the collection.

Adding data to the Vendor and Foodtype collection
Next, you can add sample data by heading to the Documents tab and clicking Add Documents, populating the fields available and clicking Create.

Vendor

document	vendorName	foodTypes
document 1	Karic’s	[document 1, document 2]
document 2	Mecurial	[document 2]

Foodtype

document	foodtype	food	foodChecked
document 1	African	[“Eba”, “Semo”]	[false, false]
document 2	Rice-dish	[“Jollof”, “Fried rice”]	[false, false]

Cloning the UI template

This section uses a UI template containing user registration and login code. Let’s clone the repository specified in the prerequisites. Check out the official GitHub docs to learn more about cloning a repository.

After cloning the UI to your PC, open it in your preferred code editor and run the command below:

flutter pub get

The lib directory tree should look like this:

lib/
├─ auth/
│  ├─ app_provider.dart
├─ model/
│  ├─ order_model.dart
│  ├─ user_model.dart
│  ├─ vendor_model.dart
├─ pages/
│  ├─ components/
│  │  ├─ HomeComponent/
│  │  │  ├─ first_column_item.dart
│  │  │  ├─ second_column_item.dart
│  │  ├─ registrationloader.dart
│  ├─ views/
│  │  ├─ modal/
│  │  │  ├─ view_one_modal.dart
│  │  ├─ view_one.dart
│  │  ├─ view_three.dart
│  │  ├─ view_two.dart
│  ├─ home.dart
│  ├─ login.dart
│  ├─ signup.dart
├─ main.dart

Connecting a Flutter project to Appwrite

Here’s how to connect a Flutter project to Appwrite for Android and iOS devices.

iOS
First, obtain the bundle ID by navigating to the project.pbxproj file (ios > Runner.xcodeproj > project.pbxproj) and searching for the PRODUCT_BUNDLE_IDENTIFIER.

Next, head to the Runner.xcworkspace folder in the application’s iOS folder in the project directory on Xcode. To select the runner target, choose the Runner project in the Xcode project navigator and find the Runner target. Next, select General and IOS 11.0 in the deployment info section as the target.

Android
For Android, copy the XML script below and paste it below the activity tag in the Androidmanifest.xml file (to find this file, head to android > app > src > main).

Note: change [PROJECT-ID] to the ID you used when creating the Appwrite project.

We will also need to set up a platform within the Appwrite console. Follow the steps below to do so.

Within the Appwrite console, select Create Platform and choose Flutter for the platform type.
Specify the operating system: in this case, Android.
Finally, provide the application and package names (found in the app-level build.gradle file).

Explaining the UI and Functionalities

Functionalities
Start by creating a folder called appConstants in the lib folder, and within the folder, create the app_constants.dart file. This file will store some important constants for the project in an Appconstants class.

class Appconstants {
    static const String projectid = "<projectID>";
    static const String endpoint = "<cloud endpoint>";
    static const String dbID = "<database ID>";
    static const String ordercollectionID = "<collection ID>";
    static const String ordercollectionID = "<collection ID>";
}

The project uses the Provider package for state management, hence the ChangeNotifier class in the app_provider.dart file.

Note: The user registration methods will not be the focal point of this article. You can do well to check out the syntax used in the method for personal learning.

The ChangeNotifier class will retrieve the list of vendors and orders, create new user orders and update the order and delivery status.

user_model.dart

vendor_model.dart

order_model.dart

app_provider.dart

In the code above, the ChangeNotifier class consists of several instance variables such as Appwrite’s client, account and databases instance, _vendorItems, _orderItems and, _user model classes and _isLoading boolean. It uses the AppProvider class constructor to initialize the _isLoading variable to true, set the _user variable to null, and call the initialize() method.

It then sets the endpoint and project ID for the client instance, creates an Account object, and sets the databases instance variable in the initialize() method. It also calls the checkUserSignIn() method, which tries to get the user account info using the _getUseraccountInfo() method. If the checkUserSignIn() method throws an error, it calls the listVendorDocument() method to list the vendor document. This is because there is no available user session at that moment.

The _getUseraccountInfo() tries to check whether a recent user session is available using the account.get() method. If true, it maps the JSON to the User model class. It lists the vendor and order documents using the listVendorDocument() and listOrderDocument() methods, respectively, while setting the _isLoading variable to false. Finally, it returns the User object.

The listVendorDocument() method lists the vendor documents from the Appwrite database and maps the document response to the Vendor model class. It sets the _vendorItems instance variable to the mapped vendor documents and the _isLoading variable to false. The listOrderDocument() method also lists the order documents from the Appwrite database and maps the document response to the Order model class. It sets the _orderItems instance variable to the mapped order documents.

createOrderDocument method creates a new order document using the createDocument method from Appwrite’s database API and notifies listeners of any changes.
updateOrderDocument method updates an order document's delivery status and delivery time.
cancelorder method cancels an order document by updating its delivery and order information.
_subscribe method subscribes to a real-time update stream of order documents and updates the corresponding order item's order status when an update event occurs.

UI (User interface)
The pages folder consists of the entirety of the user interface. The user interface is a single-page interface consisting of a split view with which, when you click the navigation, it redirects to the view linked to the click event.

The first view (view_one.dart) consists of a row of scrollable containers. These containers will list all the items in the vendor collection and their corresponding relationship with foodtypes.

When you click on any of the items, you will get an alertdialog that shows the items of the foodtype associated with the particular vendor. This is possible because of the many to many relation between vendors and foodtypes. This relation allows each vendor to have many food types and vice versa.

view_one.dart

view_one_modal.dart

Once an order is created, the order and delivery information are displayed in the order section of the UI in the view_three.dart file. The Order section of the UI contains a ListTile widget with an onTap property that displays a modal showing the order details, a button to cancel an order (by calling the cancelorder function in the ChangeNotifier class), and the delivery status.

Once the order is marked as delivered, a prompt appears in the modal asking the client if they have received their package. If the option is yes, the corresponding document in the delivery collection is updated.

view_three.dart

When you run the application, your result should look like the gifs below:

Conclusion

Appwrite’s relationship database has shown extreme versatility by providing the option of selecting different relations, which allows for building different scalable applications while being organized. By using the many to many relation, this tutorial has shown a relationship between different vendors and different food types. Similarly, by using the one to one relation, we have shown the relationship between one order and one delivery.

Resources

Getting started with Appwrite Cloud and Flutter

Femi-ige Muyiwa — Mon, 05 Jun 2023 10:52:49 +0000

Isn’t it nice to have all the traditional backend features on the web? Appwrite cloud alleviates the need to manage a server. Appwrite cloud is a backend as a service (BaaS) with all the necessary APIs to build your modern application. These APIs assist in building client and server-side applications. A use case of Appwrite is the Database API, which assists in implementing CRUD (create, read, update, delete) features in your application.

This article will show you how to use an input element to create, store, view and delete text in Flutter using Appwrite.

Prerequisites

To follow along with this tutorial, the following are required:

Xcode (with developer account for Mac users)
iOS Simulator, Android Studio, or Chrome web browser to run the application
Appwrite cloud access. It is currently in beta testing.

Creating an Appwrite project

You can apply to the Appwrite cloud to access the private beta. Once you access Appwrite’s cloud private beta, create a project and fill in the project's name and id; otherwise, use Appwrite locally with Docker or DigitalOcean droplet.

Also, you will need a database collection to perform CRUD operations. Head to Databases, select Create Database, and fill in the database name and ID. After, create a new collection by selecting Create collection and filling in the subsequent name and ID.

It is necessary to have parameters to hold data within your collection. Thus, head to the attributes section within the collection created earlier and add the attributes in the following table to the collection:

Attribute key	Attribute type
title	String
subtitle	String

Finally, head to the collection settings section and set the collection and document level permission to role:any and check all the CRUD permissions. These permissions allow anyone to create, read, update, and delete documents from the collection.

Cloning the UI template

After cloning the UI to your local storage, open it in your preferred code editor and run the command below:



flutter pub get

The lib directory tree should look like this:



    lib/
    ├─ auth/
    │  ├─ app_provider.dart
    ├─ model/
    │  ├─ documentmodel.dart
    ├─ pages/
    │  ├─ components/
    │  │  ├─ add_item_modal.dart
    │  │  ├─ edit_item_modal.dart
    │  ├─ home.dart
    ├─ main.dart

Explaining the UI and functionalities

Functionalities
Before beginning, create a file app_constants.dart to store important constants from Appwrite. Update the app_constants.dart with the code below:


 javascript
class Appconstants {
  static const String projectid = "<projectID>";
  static const String endpoint = "<cloud endpoint>";
  static const String dbID = "<database ID>";
  static const String collectionID = "<collection ID>";
}

The project utilizes the Provider package for its state management purposes. Thus, the app_provider.dart file defines a class named AppProvider that is implementing ChangeNotifier. The AppProvider class contains various methods for creating, listing, updating, and removing documents in a database using the Appwrite package.

In the code above, the AppProvider class implements the ChangeNotifier class to handle changes in your application’s state. The Client class instance, named client, is tasked with communicating with the Appwrite API.

The Account and Databases instances communicate with the account and database API. This allows you to create a user account on the client side and perform the CRUD operations.

The code above uses the initialize method to set up the client instance by setting the endpoint and project and creating an anonymous session for the user.

To utilize the CRUD operations in Appwrite, here is a summary of the document-based methods in the code:

createDocument method takes in newTitle, newSubtitle, and context parameters to create a new document in the specified database and collection. It then adds the new document to the list of documents and closes the add document screen.
listDocument method lists all the documents in the specified database and collection. It maps the returned list of documents to a list of ListItem objects and sets the _listItem variable to this list.

You can use a model class to map the JSON when getting a list of documents from Appwrite. Thus, add the code below in documentmodel.dart file


 javascript
class UserFields {
  static const String id = "\$id";
  static const String title = "title";
  static const String subtitle = "subtitle";
}
class ListItem {
  String? id;
  String? title;
  String? subtitle;
  ListItem({
    required this.id,
    required this.title,
    required this.subtitle,
  });
  ListItem.fromJson(Map<String, dynamic> json) {
    id = json[UserFields.id];
    title = json[UserFields.title];
    subtitle = json[UserFields.subtitle];
  }
  Map<String, dynamic> toJson() {
    final Map<String, dynamic> data = <String, dynamic>{};
    data[UserFields.id] = id;
    data[UserFields.title] = title;
    data[UserFields.subtitle] = subtitle;
    return data;
  }
}

the updateDocument method takes in documentId, updateTitle, and updateSubtitle parameters to update the specified document in the database with the new title and subtitle.
the removeDocument method takes in documentId and index parameters to delete the specified document from the database and remove it from the list of documents.

Overall, the AppProvider class provides a way to manage documents in an Appwrite database in a Flutter application.

UI (User interface)
The pages folder consists of the entirety of the user interface. The user interface consists of a centered container whose primary action is to display a list of created items. The Container has an ElevatedButton widget with an onPressed action displaying a modal. This displayed modal consists of a formfield (input and button) that takes values to create an item.

To perform an update action, wrap the list in an Inkwell widget and attach an onTap property that shows an editable modal that updates the list and the document on Appwrite.


 javascript
//Home.dart 

Widget appContainer(BuildContext context, state) {
  return Container(
      height: MediaQuery.of(context).size.height,
      width: MediaQuery.of(context).size.width,
      margin: const EdgeInsets.only(left: 50, right: 50, top: 50),
      decoration: BoxDecoration(
        borderRadius: BorderRadius.circular(25.0),
        color: const Color.fromARGB(238, 238, 232, 198),
      ),
      child: Column(
        crossAxisAlignment: CrossAxisAlignment.start,
        children: [
          Padding(
            padding: const EdgeInsets.all(20.0),
            child: ElevatedButton.icon(
              onPressed: () {
                // Handle the tap event by displaying a dialog box where the user can edit the data
                showDialog(
                  context: context,
                  builder: (BuildContext context) {
                    // Return the dialog widget
                    return const AddItemDialog();
                  },
                );
              },
              icon: const Icon(
                Icons.add,
                size: 24.0,
              ),
              label: const Text("Add Item"),
            ),
          ),
          items(state)
        ],
      ));
}
Widget items(state) {
  return Expanded(
      child: Container(
          child: state.listItem.isEmpty
              ? const Center(
                  child: Text(
                    "No items found",
                    style: TextStyle(fontWeight: FontWeight.w800),
                  ),
                )
              : listContainer(state)));
}
Widget listContainer(state) {
  return ListView.builder(
    itemCount: state.listItem.length,
    itemBuilder: (BuildContext context, int index) {
      return InkWell(
        onTap: () {
          // Handle the tap event by displaying a dialog box where the user can edit the data
          showDialog(
            context: context,
            builder: (BuildContext context) {
              // Return the dialog widget
              return EditListItemDialog(
                item: state.listItem[index],
                onSave: (ListItem editedItem) {
                  // Update the item in the list with the edited data
                  state.listItem[index] = editedItem;
                  // Rebuild the UI to reflect the updated data
                  setState(() {});
                },
              );
            },
          );
        },
        child: ListTile(
          title: Text(state.listItem[index].title),
          subtitle: Text(state.listItem[index].subtitle),
          trailing: IconButton(
            icon: const Icon(Icons.delete),
            onPressed: () =>
                state.removeReminder(state.listItem[index].id, index),
          ),
        ),
      );
    },
  );
}

Add item AlertDialog

Edit/Update item AlertDialog

When you run the application, your result should look like the GIF below:

Conclusion

Though still in beta testing, Appwrite Cloud is a game-changer, enabling the creation of backend functionalities swiftly and easily while providing the same benefits as using Appwrite locally. This tutorial has shown how to create CRUD features using the Appwrite cloud and Flutter.

Resources

How to email PDF results with Appwrite cloud, Mailgun and Flutter

Femi-ige Muyiwa — Wed, 26 Apr 2023 14:29:38 +0000

The cloud: a beautiful firmament containing "a visible mass of condensed water vapor floating in the atmosphere, typically high above the general level of the ground." Well, that’s what the Oxford Dictionary said, but the cloud is also a beautiful thing for developers. The cloud is a place of infinite possibilities and boundless opportunities, containing a global network of servers, all with a unique function.

That brings us to Appwrite. Isn't it great to have all the backend capabilities pushed to the web? With the Appwrite cloud, one can access all the standard backend features without installing anything. This tutorial demonstrates how to create and deploy an Appwrite function to email PDF results with Mailgun and Flutter.

What we are going to cover:
- How to create an Appwrite function
- How to deploy the function to the cloud and locally (Appwrite cloud is still in private beta)
- How to execute the function from the client side using the client SDK

Prerequisites

To folllow along with this tutorial, the following are required:

Xcode (with developer account for Mac users).
iOS Simulator, Android Studio, or Chrome web browser to run the application.
Appwrite CLI installed.
Appwrite cloud access. It is currently in beta testing.
Mailgun account access. Here is a link to sign up.
An Appwrite instance running on either Docker, DigitalOcean droplet, or Gitpod. You’ll need this because we will also show how to deploy the function locally.

Account and project setup

In this section, we’ll cover how to set up the following:

Mailgun account
Appwrite cloud account
Appwrite project

Setting up a Mailgun account
To sign up for Mailgun, follow these steps:

Start by signing up for a new account
Fill out the signup form (name, email address, password) and add the credit card information required
Click on the Create Account button to submit the form
Check your email inbox for a confirmation message from Mailgun
Follow the instructions in the email to verify the account and complete the signup process.

After completing the signing up and verification processes, we must add and verify our domain to send emails. Here are some places to purchase a domain:

Next, follow the steps below to verify the DNS settings.

After adding the domain or subdomain, open your DNS provider and add the two TXT DNS records provided.
Next, add the CNAME record to allow Mailgun to track clicks and opens.
Finally, add the MX records.

Once we’ve added the records, our domain will be verified.

Setting up an Appwrite cloud account
To sign up for Appwrite, follow these steps:

After registering for the Appwrite cloud beta, you will receive an invite via email.
Click on the invite link and follow the registration process. That’s it!

Setting up the Appwrite project
After creating a cloud account, we'll create a platform and an Appwrite project. Let's start by doing the following:

Note: This process is the same both locally and on the cloud

For the Appwrite cloud, you’ll be redirected to the Create project page after successful sign-in, but locally, we will need to load up the Appwrite console. Thus, open your browser and enter the IP address or hostname.
Next, choose the Create Project option and fill in the name and ID (ID can be automatically generated).
Next, we will create a collection to hold the URL string to an online PDF file. Thus, we will create one string attribute called url.

url
https://www.w3.org/WAI/WCAG21/working-examples/pdf-table/table.pdf

Finally, set the collection-level CRUD permission for both collections to Any and select all the options.

Creating an Appwrite project

Next, we’ll create two projects to achieve our desired result. For the first project, we will create an Appwrite function using the Appwrite CLI (command line interface). The second project will be a Flutter project to communicate with the function created earlier after deployment.

Thus, run the command below to create two directories for the projects, respectively:

mkdir mailgun_function && mkdir mailgun_flutter_app

Project one (Creating a dart Appwrite function)
We will start by enabling the Dart runtime in Appwrite, as it is not naturally included. Head to the default Appwrite directory (when installing locally) and make some edits to the .env file. Add dart 2-17 to the _APP_FUNCTIONS_RUNTIMES and _APP_FUNCTIONS_ENVS variables like this:

_APP_FUNCTIONS_RUNTIMES=node-16.0,php-8.0,python-3.9,ruby-3.0,dart-2.17
_APP_FUNCTIONS_ENVS=node-16.0,php-7.4,python-3.9,ruby-3.0,dart-2.17

After making the changes, we will run the command below from the default Appwrite directory in our terminal:

docker compose up -d --force-recreate

Note: Make sure Docker is running before inputing the command above.

Next, we will head to the mailgun_function folder and log into the Appwrite server using its CLI.

appwrite login

You will receive a prompt to add the email and password you used when registering the local instance. Next, create a new project or specify an existing one. We can do that using the following command:

appwrite init project

We will then follow the prompt on how to set the project up.

Next, we must create an Appwrite function. Therefore, we will do that by running the command:

appwrite init function

Similarly to the previous commands, you will receive a prompt directing you on how to set this up. Once done setting the function up, the command creates a starter project.

Now, head to the pubspec.yaml file and add the http package.

http: ^0.13.3

In the main.dart file, replace the existing code with the one below:

The snippet above does the following:

Imports the required dependencies
Requests custom payload data when executing the function
Passes the payload using Uri.parse(), converts it back to a file, and stores it in temporary storage
Configures http mail client by specifying the API Key, and parses the domain URL, to emails, from emails, email content, and email subject
Sends the email using the configured client

Deploy the function
For this tutorial, we will show how to deploy the functions both locally and on the cloud.

Cloud

We need to create a function on the Appwrite cloud console to deploy on the cloud. Thus, head to the function section, click Create Function, fill in the name and ID, and select the preferred runtime (dart — 2.17).

Next, select Create Deployment and head to the manual deployment. In the manual deployment, we will need to add a tar.gz fileand specify the entrypoint for our project. So, head to the folder containing the pubspec.lock file and run the command below:

tar -czf code.tar.gz --exclude code.tar.gz .

This command will create a tar.gz file that can be uploaded to Appwrite. So we will head back to the cloud console, specify the entry point as lib/main.dart, then add the tar.gz file and click create.

Locally

We can do this by navigating to the folder containing the appwrite.json file and then running the command below:

appwrite deploy function

Note: Within the function section, head to the settings section and specify Execute Access to any.

Project two (Creating a Flutter application)

Head to the second folder we created earlier and run the following command

git clone https://github.com/muyiwexy/Appwrite_Cloud_MailGun.git && cd Appwrite_Cloud_MailGun

This command will clone a flutter template within the current directory and change it to the Appwrite_Cloud_MailGun directory.

The template above contains the following:

The main.dart file, which contains:
- A button widget to execute the function on the client side
- A snackbar to indicate success or failure
The utils.dart file, which contains a json serialize to map the JSON object received from the database to a model class

Next, we must install and connect our Flutter application to Appwrite.

First, add Appwrite to the dependencies section of the pubspec.yaml file, like the image below:

Alternatively, we can use a terminal by typing the command below:

flutter pub add appwrite

Here’s how to connect a Flutter project to Appwrite for Android and iOS devices.

iOS

First, obtain the bundle ID by navigating to the project.pbxproj file (ios > Runner.xcodeproj > project.pbxproj) and searching for the PRODUCT_BUNDLE_IDENTIFIER.

Android

For Android, copy the XML script below and paste it below the activity tag in the Androidmanifest.xml file (to find this file, head to android > app > src > main).

Note: change [PROJECT-ID] to the ID you used when creating the Appwrite project.

We will also need to set up a platform within the Appwrite console. Follow the steps below to do so.

Within the Appwrite console, select Create Platform and choose Flutter for the platform type.
Specify the operating system: in this case, Android.
Finally, provide the application and package names (found in the app-level build.gradle file).

Running the Flutter project
Once we are done with the above, run the following command:

flutter pub get

This command obtains all the dependencies in the pubspec.yaml file. Next, run the flutter run command to run the project on any desired platform.

Here is the result of the code:

Conclusion

In conclusion, emailing PDF results with Appwrite Cloud, Mailgun, and Flutter is a straightforward process that can be accomplished with simple steps. By using Appwrite Cloud to generate and retrieve PDF results, Mailgun to send email notifications, and Flutter to handle the user interface and app logic, developers can easily create a robust and efficient system for emailing PDF results to users.

DEV Community: Femi-ige Muyiwa

Run Postgres For Free: Top 3 Options

Why PostgreSQL?

Managed Postgres Comparison Table

Neon

Aiven

Render

Final Thoughts

Resources

5 Must-Watch Tutorials to Build Your SaaS App in 2025

1. Building a Finance Manager SaaS Platform

2. Build a SaaS App With Django, Stripe, Neon PostgreSQL, TailwindCSS, GitHub Actions

3. How To Build A $1,000,000 SaaS In 7 Hours

4. SaaS Automation Builder

5. SaaS AI Chatbot - Email Marketing, Next.js, Clerk, Neon, Uploadcare, Cloudways, Bun, Stripe, Pusher

Final Thoughts

Building AI-powered mobile app with RAG, pgvector & OpenAI

Prerequisites

Breaking down the technologies

Setting up a Neon Postgres project

Creating the Flutter application

Cloning the UI template

Implementing the RAG technique

Indexing

Retrieval

Handling the abstraction, state management, and dependency injection

rag with openai | Opentape

Conclusion

Resources

How to set up Auth0 and Neon Postgres in Next.js App Router

Prerequisites

Setting up a Neon Postgres project

Creating a Next.js application and installing dependencies

Configuring Auth0 application settings

Connecting NextJs to Auth0

Creating an API route for Auth0

Building the pages and routes using Next.js and Auth0

auth0 next js | Opentape

Setting up Neon as a custom database on Auth0

Creating users database table on Neon

Conclusion

Resources

Basics of RAG with pgVector and Langchain

Demystifying some concepts

Creating a Neon database

Creating the Flutter application

Indexing

Retrieval

neon and langchain | Opentape

Conclusion

Resources

How to add push notifications to your Appwrite Cloud apps

Getting started

Creating projects

Creating a Firebase project

Creating an Appwrite project

Creating a Flutter project

Setting up Firebase Cloud Messaging

Generating a private key

Creating a Realtime database

Setting up Appwrite

Creating an Appwrite database

Creating an Appwrite Cloud function FCM template

Customzing the FCM template code

Building the Flutter project

Cloning the application for the notification device

iOS

Android

Cloning the application for the admin device

Conclusion

Resources

Implementing OAuth2 Clients with Flutter made simple

Prerequisites

Creating an Appwrite project

Getting started

Conclusion

SMS authentication using Appwrite and Twilio

Setting up Twilio in Appwrite

Conclusion

Resources

Creating `users` database table on Neon