DEV Community: Neon

Interact with Neon APIs using Claude Code through natural language

Pedro Figueiredo — Fri, 29 Aug 2025 15:20:38 +0000

Imagine adjusting your database schema simply by describing the change in plain English. This is possible by combining Claude Code with the Neon MCP Server.

This guide demonstrates how to use Claude Code's command-line interface and Neon's MCP server to perform database migrations in your Neon project.

The Neon MCP Server grants powerful database management capabilities through natural language requests. Always review and authorize actions requested by the LLM before execution. Ensure that only authorized users and applications have access to the Neon MCP Server.

The Neon MCP Server is intended for local development and IDE integrations only. We do not recommend using the Neon MCP Server in production environments. It can execute powerful operations that may lead to accidental or unauthorized changes.

For more information, see MCP security guidance.

Key components

Let's break down the key components in this setup:

Claude Code: Claude Code is Anthropic's official CLI tool that supports Model Context Protocol (MCP) for interfacing with external tools (APIs, databases, etc.)
Neon MCP Server: Neon's MCP server acts as a bridge between MCP clients like Claude Code and Neon's API, letting you work with Neon databases using natural language commands.
Model Context Protocol (MCP): MCP is a lightweight communication standard that allows Claude Code and Neon MCP Server to work together.

Setting up Neon MCP Server in Claude Code

You have two options for connecting Claude Code to the Neon MCP Server:

Remote MCP Server (Preview): Connect to Neon's managed MCP server using OAuth for authentication. This method is more convenient as it eliminates the need to manage API keys in Claude Code. Additionally, you will automatically receive the latest features and improvements as soon as they are released.
Local MCP Server: Run the Neon MCP server locally on your machine, authenticating with a Neon API key.

Prerequisites

Before you begin, ensure you have the following:

Claude Code: Ensure you have Claude Code installed. Visit docs.anthropic.com/en/docs/claude-code for installation instructions.
Neon API Key (for Local MCP server): After signing up, get your Neon API Key from the Neon console. This API key is needed to authenticate your application with Neon. For instructions, see Manage API keys.

Keep your Neon API key secure, and never share it publicly. It provides access to your Neon projects.

Node.js (>= v18) and npm: Ensure Node.js (version 18 or later) and npm are installed. Download them from nodejs.org.

Option 1: Setting up the Remote Hosted Neon MCP Server

Note: By default, the Remote MCP Server connects to your personal Neon account. To connect to an organization's account, you must authenticate with an API key. For more information, see API key-based authentication.

This method uses Neon's managed server and OAuth authentication.

Open your terminal.
Add the Neon MCP server to Claude Code with the following command:

   claude mcp add --transport http neon https://mcp.neon.tech/mcp

Start a new session of claude to trigger the OAuth authentication flow:

   claude

You can also trigger authentication with /mcp within Claude Code.

If you prefer to authenticate using a Neon API key, provide the Authorization header to the mcp add command:

claude mcp add --transport http neon https://mcp.neon.tech/mcp \
    --header "Authorization: Bearer <YOUR_NEON_API_KEY>"

Replace <YOUR_NEON_API_KEY> with your actual Neon API key which you obtained from the prerequisites section.

The remote hosted MCP server is in preview due to the new OAuth MCP specification, expect potential changes as we continue to refine the OAuth integration.

Option 2: Setting up the Local Neon MCP Server

This method runs the Neon MCP server locally on your machine, using a Neon API key for authentication.

Open your terminal.
Add the Neon MCP server to Claude Code with the following command, replacing <YOUR_NEON_API_KEY> with your actual Neon API key:

   claude mcp add neon -- npx -y @neondatabase/mcp-server-neon start "<YOUR_NEON_API_KEY>"

Replace <YOUR_NEON_API_KEY> with your actual Neon API key which you obtained from the prerequisites section.

Start a new Claude Code session with the claude command and start using the Neon MCP server:

   claude

You've now configured the Neon MCP Server in Claude Code and can manage your Neon Postgres databases using AI.

Verification

Now that you have the Neon MCP server set up either remotely or locally, you can verify the connection and test the available tools.

Start Claude Code:

   claude

Type /mcp to see the available MCP servers and tools.
Try out a Neon MCP Server tool by typing a query like List my Neon projects to see your projects and verify the connection.

Supported Actions (Tools)

The Neon MCP Server provides the following actions, which are exposed as "tools" to MCP Clients. You can use these tools to interact with your Neon projects and databases using natural language commands.

Project management:
list_projects: Retrieves a list of your Neon projects, providing a summary of each project associated with your Neon account. Supports limiting the number of projects returned (default: 10).
describe_project: Fetches detailed information about a specific Neon project, including its ID, name, and associated branches and databases.
create_project: Creates a new Neon project in your Neon account. A project acts as a container for branches, databases, roles, and computes.
delete_project: Deletes an existing Neon project and all its associated resources.
list_organizations: Lists all organizations that the current user has access to. Optionally filter by organization name or ID using the search parameter.

Branch management:
create_branch: Creates a new branch within a specified Neon project. Leverages Neon's branching feature for development, testing, or migrations.
delete_branch: Deletes an existing branch from a Neon project.
describe_branch: Retrieves details about a specific branch, such as its name, ID, and parent branch.
list_branch_computes: Lists compute endpoints for a project or specific branch, including compute ID, type, size, and autoscaling information.

SQL query execution:
get_connection_string: Returns your database connection string.
run_sql: Executes a single SQL query against a specified Neon database. Supports both read and write operations.
run_sql_transaction: Executes a series of SQL queries within a single transaction against a Neon database.
get_database_tables: Lists all tables within a specified Neon database.
describe_table_schema: Retrieves the schema definition of a specific table, detailing columns, data types, and constraints.
list_slow_queries: Identifies performance bottlenecks by finding the slowest queries in a database. Requires the pg_stat_statements extension.

Database migrations (schema changes):
prepare_database_migration: Initiates a database migration process. Critically, it creates a temporary branch to apply and test the migration safely before affecting the main branch.
complete_database_migration: Finalizes and applies a prepared database migration to the main branch. This action merges changes from the temporary migration branch and cleans up temporary resources.

Query performance tuning:
explain_sql_statement: Analyzes a SQL query and returns detailed execution plan information to help understand query performance.
prepare_query_tuning: Identifies potential performance issues in a SQL query and suggests optimizations. Creates a temporary branch for testing improvements.
complete_query_tuning: Finalizes and applies query optimizations after testing. Merges changes from the temporary tuning branch to the main branch.

Neon Auth:
provision_neon_auth: Provisions Neon Auth for a Neon project. Sets up authentication infrastructure by creating an integration with Stack Auth (@stackframe/stack).
These actions enable any MCP client like Claude Code to interact with various functionalities of the Neon platform via the Neon API. Certain tools, especially database migration ones, are tailored for AI agent and LLM usage, leveraging Neon's branching for safe preview and commit.

These actions enable any MCP client like Claude Code to interact with various functionalities of the Neon platform via the Neon API. Certain tools, especially database migration ones, are tailored for AI agent and LLM usage, leveraging Neon's branching for safe preview and commit.

Development Workflow: Adding a Column with Claude Code and Neon MCP

Let's walk through a typical development scenario: Quickly adding a column for prototyping using natural language within Claude Code. The following chat log demonstrates a real interaction with Claude Code and the Neon MCP server.

Scenario: During development, you decide to track timestamps for entries in your playing_with_neon table. You want to quickly add a created_at column.

For your security, review the tool's purpose before permitting the operation to proceed. Remember that LLMs can sometimes produce unexpected results, so careful monitoring is always recommended.

Here's the conversation log between the user and Claude Code:

User: In my neon project id: round-salad-44063611 list all the tables
Claude Code: I'll help you list all the tables in your Neon project. I'll use the default database name 'neondb' since none was specified.
> Called MCP Tool (get_database_tables)
> I found one table in your database:
> playing_with_neon (in the public schema)
> Would you like to see more details about this table's structure or perform any other operations with it?

User: Can you show me first 5 rows?
Claude Code: I'll query the first 5 rows from the playing_with_neon table.
> Called MCP Tool (run_sql)
> Here are the first 5 rows from the table. The table has three columns:
> id (sequential numbers)
> name (string values)
> value (decimal numbers)
> Would you like to perform any other operations with this data?

User: can you add a created_at column?
Claude Code: I'll help you add a created_at column to the playing_with_neon table. I'll set it up with a timestamp data type and set it to automatically record the creation time of new rows.
> Called MCP Tool (prepare_database_migration)
> Let me verify the change in the temporary branch to make sure it was applied correctly.
> Called MCP Tool (run_sql)
> I've added the created_at column to the playing_with_neon table in a temporary branch. Would you like to commit this migration to the production branch?
> Migration Details:
> Migration ID: a65babcd-d1cc-401f-89b5-762570570102
> Temporary Branch Name: br-silent-cherry-a1eqrl6i
> Temporary Branch ID: br-silent-cherry-a1eqrl6i
> Migration Result: Success
> Please let me know if you'd like to proceed with applying this change to the production branch.

User: yeah do it
Claude Code: I'll complete the migration and apply the changes to the production branch.
> Called MCP Tool (complete_database_migration)
> The migration has been successfully completed! The created_at column has been added to your table in the production branch, and the temporary branch has been cleaned up.

Key takeaways:

Natural language interaction: You can use simple, conversational English to interact with your database.
Step-by-step guidance: Claude Code confirms each step and provides details like branch names and migration IDs.
MCP Tools in Action: The underlying MCP tools (get_database_tables, run_sql, prepare_database_migration, complete_database_migration) illustrate the automated process.
Branching for safety: The agent automatically creates a temporary branch for schema changes. The user will be prompted to confirm the migration before applying it to the production branch.

You can verify the changes in your Neon Console or by querying the database directly.

Bonus: Creating a project and branch

Beyond schema changes, you can also manage your Neon projects and branches directly within Claude Code using natural language commands. This can be useful for quickly setting up a new development or test environment.

Creating a new project

Let's say you want to create a new Neon project named "my-new-project". You can simply tell Claude Code:

User: Create a Neon project named "my-new-project"
Claude Code: I'll help you create a new Neon project with the specified name.
> Called MCP Tool (create_project)
> Great! I've created a new Neon project for you with the name "my-new-project". Here are the important details:
> Project ID: orange-dawn-33739962
> Default branch: main
> Default database: neondb
> The project is ready to use. You can start creating tables and working with your database right away.

Claude Code will then use the create_project MCP tool to initiate the project creation. It will provide you with a project ID and name.

Creating a New Branch

Once you have a project, you can easily create new branches for development or experimentation. For example, to create a branch named "feature-x" in your "my-new-project" project:

User: Create a branch named "feature-x"
Claude Code: Perfect! I've created a new branch named "feature-x". Here are the details:
> Branch ID: br-cold-mountain-a523ka2w
> Branch name: feature-x
> Parent branch ID: br-odd-pine-a5y53krm
> The branch has been created successfully and is ready for use.

Claude Code will use the create_branch MCP tool to create the branch and provide you with the branch name and ID. Notice how we don't need to specify the project ID, as Claude Code remembers the active project context.

Conclusion

Claude Code combined with the Neon MCP Server, whether using the Remote Hosted (Preview) option or the Local Server setup, lets you use natural language to interact with your database and take advantage of Neon's branching capabilities for fast iteration. This approach is ideal for quickly testing database ideas and making schema changes during development.

Resources

Build Internal Tools Better: Using Neon, StackAuth, and Vercel

Sam シ — Thu, 10 Jul 2025 15:33:24 +0000

Almost every tech company, from small startups to Fortune 500 enterprises relies on internal tools. Larger organizations often have dedicated services and structured procedures in place, but for many others, internal tools are... messy. They often take the form of an insecure setup hosted behind a VPN or worse, an overly complex solution cobbled together from a surprising number of often times costly AWS services.

But if you’re like me, you just want to share tools easily, securely, cheaply, and without feeling like you’re managing a full-fledged production application (that's already your 9 to 5).

That's what drove me to create this template.

Built using Neon, StackAuth, and Vercel's free plans, it provides a secure and scalable foundation, enabling you to quickly launch your next internal tool in minutes: https://github.com/sam-harri/internal_tooling_neon_stack

It comes preconfigured with a Postgres database, API routes, authentication, authorization, and a built-in admin panel – everything you need to get an application running fast.

Getting Started with the Template

Using the template, you can get your internal tool up and running in minutes, and all you need are Neon, Stack Auth, and Vercel accounts.

git clone https://github.com/sam-harri/internal_tooling_neon_stack.git

Then open that new project in your editor of choice, and install the dependencies:

npm install

Next, set up the environment variables by copying over the .env.example to a new .env file. Fill these in with the connection string from your Neon database project in the console, and the Stack Auth keys from the Auth tab.

Since the template comes with some admin logic already, you’ll need to sync your database by applying the schema:

npx drizzle-kit push

Set Up Roles and Claim Your Project

Head to the Auth tab in the Neon console and claim the project with your Stack Auth account. From there, create two project roles: admin and user, where user is contained within admin. Your Project Permissions section should look like this afterward:

Once that’s done, boot up the project locally, log in, and head over to /setup to claim your admin privileges.

You can then deploy your app with Vercel. Don’t forget to add your Vercel-provided domain to the allowed domains in Neon’s Auth tab, or your custom domain if you're using one.

Authentication and Access Control with StackAuth

You obviously don't want just anyone to be able to access your internal tools. Luckily, Neon comes with an auth solution supporting social and password sign-on, role-based access control, and much more.

In this template, only authorized users can access the tools. The application admin can whitelist domains, allowing anyone with an email from a specific domain (e.g. yourcompany.tld) and a verified email to sign up and get access. Likewise, you can grant and revoke access to specific external email addresses, perfect for contractors or partners. Emails can also be explicitly blocked even if their domain is whitelisted.

Managing access is straightforward via the built-in admin panel at /tools/admin. Admins can assign or revoke privileges to other users, giving you complete control over your application's access policies. These rules are meant to cover most of the common scenarios you might face, but since it’s a template, you can easily update it to add features like regex validation or subdomain whitelisting.

Database and Logic with Neon and Next.js

Internal tools inherently require business logic and data storage. Neon provides a free serverless Postgres database, along with all the most popular Postgres extensions, and offers a serverless database driver that’s perfect for single-shot queries in Next.js serverless functions.

For business logic, you can use Next’s server actions and API routes without the overhead of managing and deploying a separate API—ideal for simple internal tools. Vercel makes hosting your Next.js app painless, even allowing you to attach a custom domain so that you can host it at something like yourtool.yourcompany.tld.

Customize the App for Your Team

Customization is designed to be simple. Just fill out your company-specific details in config/app.ts, then add your custom tools to the app/tools directory and declare them within the configuration file. To get started, play around with app/tools/tool1/page.tsx

Get Started

Clone the template, connect your Neon and StackAuth accounts, and deploy to Vercel. Your next internal tool is just a few commands away!

Building Intelligent Search with AI Embeddings, Neon, and pgvector

Bobby — Fri, 20 Jun 2025 17:21:55 +0000

Traditional text search relies on exact keyword matches, which often misses the semantic meaning behind queries.

When someone searches for "car maintenance," they might also be interested in results about "vehicle servicing" or "auto repair", but keyword-based search won't make these connections.

AI embeddings solve this problem by converting text into high-dimensional vectors that capture semantic meaning. Words and phrases with similar meanings cluster together in this vector space, enabling search systems that understand context and intent rather than just matching exact words.

The pgvector extension brings vector similarity search directly into PostgreSQL, letting you store embeddings alongside your regular data and perform complex semantic searches with simple SQL queries. Combined with Neon's serverless PostgreSQL, you can build intelligent search systems that scale automatically with your application's needs.

In this guide, you'll learn how to build a semantic search system that can power document search and content recommendations using OpenAI embeddings stored in Neon with pgvector.

What you'll build

By the end of this guide, you'll have:

An understanding of AI embeddings and how they improve search
A Neon database configured with the pgvector extension
An embedding generation service powered by OpenAI's API
A document search system that understands semantic similarity

Prerequisites

To follow along with this guide, you'll need:

A Neon account with a project
An OpenAI API key
Node.js 20.x or later installed
Basic familiarity with SQL and REST APIs
Understanding of JavaScript promises and async/await

Understanding AI embeddings and vector search

Before diving into the implementation, let's understand how AI embeddings work and why they're great for search.

What are embeddings?

Embeddings are numerical representations of text that capture semantic meaning in high-dimensional space. When you send text to OpenAI's embedding API, it returns an array of floating-point numbers (typically 1,536 dimensions for the text-embedding-3-small model) that represents the "meaning" of that text.

Here's a simplified example of how embeddings work:

// These texts have similar meanings
const text1 = "The cat jumped over the fence";
const text2 = "A feline leaped across a barrier";

// Their embeddings would be mathematically similar
const embedding1 = [-0.02, 0.15, -0.08, ...]; // 1,536 numbers
const embedding2 = [-0.01, 0.16, -0.09, ...]; // Similar values

The key insight is that semantically similar texts produce similar embedding vectors. This enables search systems that can find relevant content even when it doesn't share exact keywords with the query.

Why pgvector?

pgvector extends PostgreSQL with vector data types and similarity search operations. Instead of moving your data to specialized vector databases, you can store embeddings alongside your existing relational data and perform vector similarity searches with SQL:

-- Find documents most similar to a query embedding
SELECT title, content,
       embedding <-> $1 as distance
FROM documents
ORDER BY embedding <-> $1
LIMIT 5;

The <-> operator calculates the distance between vectors, with smaller distances indicating higher similarity.

Now let's build a system that puts these concepts to work.

Setting up pgvector on Neon

We'll start by enabling pgvector on your Neon database and creating the necessary tables for our search system.

First, create a new Neon project optimized for vector operations:

Navigate to the Neon Console
Click "New Project"
Name your project "semantic-search-system"
Choose a region close to your users
Select at least 1 CU for compute size (vector operations can be CPU-intensive)

Once your project is created, we need to enable the pgvector extension. Connect to your database and run this SQL:

-- Enable the pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Verify the extension is working
SELECT extversion FROM pg_extension WHERE extname = 'vector';

This command adds vector data types and similarity functions to your PostgreSQL database. You should see confirmation that the vector extension is now available.

Next, we'll create the database schema for our semantic search system. This includes tables for documents and their embeddings, along with indexes for fast vector similarity search.

-- Table for storing documents with their embeddings
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    title TEXT NOT NULL,
    content TEXT NOT NULL,
    category VARCHAR(100),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    embedding vector(1536) -- OpenAI's text-embedding-3-small produces 1536-dimensional vectors
);

-- Create indexes for fast vector similarity search
CREATE INDEX documents_embedding_idx ON documents USING ivfflat (embedding vector_cosine_ops);

The vector(1536) data type stores 1,536-dimensional vectors, matching OpenAI's embedding size. The ivfflat indexes enable fast approximate nearest neighbor searches using cosine similarity, this is crucial for performance when searching through thousands of embeddings.

Building an embedding generation service

Now we'll create a Node.js service that handles the complexities of generating embeddings with OpenAI's API. This service will be the bridge between your text content and the vector representations stored in the database.

Let's set up the project structure and install the necessary dependencies:

mkdir semantic-search-service
cd semantic-search-service
npm init -y
npm install openai pg dotenv express cors

Create a .env file to securely store your API credentials:

OPENAI_API_KEY=your_openai_api_key_here
DATABASE_URL=postgresql://user:password@ep-abc123.region.aws.neon.tech/neondb

Now let's create an embedding service that handles OpenAI API interactions. This service will preprocess text, generate embeddings, and handle errors gracefully:

// embedding.js
const OpenAI = require('openai');
require('dotenv').config();

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

class EmbeddingService {
  constructor() {
    this.model = 'text-embedding-3-small'; // Cost-effective and performant
    this.maxTokens = 8191; // Maximum tokens for this model
  }

  async generateEmbedding(text) {
    try {
      // Clean and prepare text for embedding generation
      const cleanText = this.preprocessText(text);

      const response = await openai.embeddings.create({
        model: this.model,
        input: cleanText,
      });

      return response.data[0].embedding;
    } catch (error) {
      console.error('Error generating embedding:', error);
      throw new Error(`Failed to generate embedding: ${error.message}`);
    }
  }

  preprocessText(text) {
    // Remove extra whitespace and normalize the text
    let cleaned = text.trim().replace(/\s+/g, ' ');

    // Truncate if too long (rough estimate: 1 token ≈ 4 characters)
    const maxChars = this.maxTokens * 3; // Conservative estimate
    if (cleaned.length > maxChars) {
      cleaned = cleaned.substring(0, maxChars) + '...';
    }

    return cleaned;
  }

  // Create optimized text for embedding generation
  createDocumentText(title, content, category = '') {
    return `Title: ${title}\nCategory: ${category}\nContent: ${content}`;
  }
}

module.exports = EmbeddingService;

This service handles a few important tasks: it preprocesses text to ensure it fits within OpenAI's token limits, combines document fields into optimized text for better embeddings, and provides error handling for API calls. The createDocumentText method is particularly important because it structures the input text in a way that produces better semantic embeddings.

As of the time of writing, OpenAI's text-embedding-3-small model is the newest and most cost-effective for generating embeddings.

Creating a document management system

Now we'll build a service that manages documents in our database and automatically generates embeddings for each document. This service will handle both individual documents and batch operations.

Let's create the document service that ties together our database and embedding generation:

// document-service.js
const { Pool } = require('pg');
const EmbeddingService = require('./embedding');
require('dotenv').config();

class DocumentService {
  constructor() {
    this.pool = new Pool({
      connectionString: process.env.DATABASE_URL,
      ssl: true,
      max: 20,
    });
    this.embeddingService = new EmbeddingService();
  }

  async addDocument(title, content, category = null) {
    const client = await this.pool.connect();

    try {
      await client.query('BEGIN');

      // Combine document fields into optimized text for embedding
      const fullText = this.embeddingService.createDocumentText(title, content, category || '');

      // Generate embedding using OpenAI API
      console.log(`Generating embedding for document: ${title}`);
      const embedding = await this.embeddingService.generateEmbedding(fullText);
      const formattedEmbedding = `[${embedding.join(',')}]`;

      // Store document with its embedding in the database
      const result = await client.query(
        `
        INSERT INTO documents (title, content, category, embedding)
        VALUES ($1, $2, $3, $4)
        RETURNING id, title, created_at
      `,
        [title, content, category, formattedEmbedding]
      );

      await client.query('COMMIT'); // Complete transaction

      console.log(`Document added successfully: ${result.rows[0].id}`);
      return result.rows[0];
    } catch (error) {
      await client.query('ROLLBACK'); // Undo changes on error
      console.error('Error adding document:', error);
      throw error;
    } finally {
      client.release(); // Return connection to pool
    }
  }

  async searchDocuments(query, limit = 10, category = null) {
    try {
      // First, convert the search query into an embedding
      const fullText = this.embeddingService.createDocumentText(query, '', category || '');
      const queryEmbedding = await this.embeddingService.generateEmbedding(fullText);

      // Format embedding as a pgvector-compatible string
      const formattedEmbedding = `[${queryEmbedding.join(',')}]`;

      // Start building the SQL query
      let sql = `
            SELECT 
                id,
                title,
                content,
                category,
                created_at,
                1 - (embedding <=> $1) as similarity_score
            FROM documents
            WHERE 1 - (embedding <=> $1) > 0.3
            `;

      const params = [formattedEmbedding];

      // Add category filter if needed
      if (category) {
        sql += ` WHERE category = $2`;
        params.push(category);
      }

      // Append ORDER BY and LIMIT clauses
      sql += ` ORDER BY embedding <=> $1 LIMIT $${params.length + 1}`;
      params.push(limit);

      const result = await this.pool.query(sql, params);

      // Format results
      return result.rows.map((row) => ({
        ...row,
        similarity_score: parseFloat(row.similarity_score.toFixed(4)),
        preview: row.content.length > 200 ? row.content.substring(0, 200) + '...' : row.content,
      }));
    } catch (error) {
      console.error('Error searching documents:', error);
      throw error;
    }
  }

  async getDocumentById(id) {
    try {
      const result = await this.pool.query(
        `
        SELECT id, title, content, category, created_at
        FROM documents 
        WHERE id = $1
      `,
        [id]
      );

      return result.rows[0] || null;
    } catch (error) {
      console.error('Error getting document:', error);
      throw error;
    }
  }
}

module.exports = DocumentService;

This service provides the core functionality for our semantic search system. The addDocument method generates embeddings and stores them alongside the document data, while searchDocuments performs the actual semantic search by converting queries to embeddings and finding the most similar documents using pgvector's distance operators.

The key insight here is the <=> operator in the SQL query, this calculates cosine distance between vectors, with smaller values indicating higher similarity. We convert this to a similarity score between 0 and 1 for easier interpretation.

The getDocumentById method retrieves a specific document by its ID, allowing applications to fetch full content when needed.

Building the search API

With our document service in place, let's create an Express API that exposes our document search capabilities to applications.

This API will provide endpoints for adding documents and performing semantic searches.

// server.js
const express = require('express');
const cors = require('cors');
const DocumentService = require('./document-service');
require('dotenv').config();

const app = express();
app.use(express.json());
app.use(cors());

const documentService = new DocumentService();

// Add a single document with automatic embedding generation
app.post('/documents', async (req, res) => {
  try {
    const { title, content, category } = req.body;

    // Validate required fields
    if (!title || !content) {
      return res.status(400).json({
        error: 'Title and content are required',
      });
    }

    // Add document and generate embedding
    const result = await documentService.addDocument(title, content, category);
    res.status(201).json(result);
  } catch (error) {
    console.error('Error adding document:', error);
    res.status(500).json({
      error: 'Failed to add document',
      message: error.message,
    });
  }
});

// Perform semantic search across all documents
app.post('/search', async (req, res) => {
  try {
    const { query, limit = 10, category } = req.body;

    if (!query) {
      return res.status(400).json({
        error: 'Search query is required',
      });
    }

    const startTime = Date.now();

    // Execute semantic search using vector similarity
    const results = await documentService.searchDocuments(query, limit, category);

    const searchTime = Date.now() - startTime;

    res.json({
      query,
      results,
      count: results.length,
      search_time_ms: searchTime,
    });
  } catch (error) {
    console.error('Error searching documents:', error);
    res.status(500).json({
      error: 'Search failed',
      message: error.message,
    });
  }
});

// Get a specific document by ID
app.get('/documents/:id', async (req, res) => {
  try {
    const { id } = req.params;
    const document = await documentService.getDocumentById(parseInt(id));

    if (!document) {
      return res.status(404).json({ error: 'Document not found' });
    }

    res.json(document);
  } catch (error) {
    console.error('Error getting document:', error);
    res.status(500).json({
      error: 'Failed to get document',
      message: error.message,
    });
  }
});

// Health check endpoint
app.get('/health', async (req, res) => {
  try {
    // Test database connection
    await documentService.pool.query('SELECT 1');
    res.json({ status: 'healthy', database: 'connected' });
  } catch (error) {
    res.status(500).json({ status: 'unhealthy', database: 'disconnected' });
  }
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Semantic search service running on port ${PORT}`);
});

This API provides clean, RESTful endpoints for our semantic search functionality. The /search endpoint is the heart of the system, it takes natural language queries and returns semantically relevant documents, even when there are no exact keyword matches.

Notice how we measure and return the search time. This helps you monitor performance as your document collection grows.

Testing the semantic search system

With our API in place, now let's test our semantic search system to see how it finds relevant documents based on meaning rather than exact keywords. We'll add sample documents and run various search queries.

Create a test script to populate your database with sample content and test the search functionality:

// test-search.js
const axios = require('axios');

const baseURL = 'http://localhost:3000';

async function testSemanticSearch() {
  try {
    console.log('Adding sample documents to test semantic search...\n');

    // Add diverse sample documents across different topics
    const sampleDocuments = [
      {
        title: 'Getting Started with Machine Learning',
        content:
          'Machine learning is a subset of artificial intelligence that enables computers to learn and make decisions from data without being explicitly programmed. This guide covers the basics of supervised learning, unsupervised learning, and neural networks.',
        category: 'technology',
      },
      {
        title: 'Healthy Cooking Tips',
        content:
          "Eating nutritious meals doesn't have to be complicated. Focus on fresh ingredients, reduce processed foods, and try cooking methods like steaming and grilling. Meal prep can save time and help maintain a balanced diet.",
        category: 'health',
      },
      {
        title: 'Remote Work Best Practices',
        content:
          'Working from home requires discipline and good habits. Set up a dedicated workspace, maintain regular hours, and use collaboration tools effectively. Communication with team members is crucial for remote success.',
        category: 'productivity',
      },
      {
        title: 'Understanding Neural Networks',
        content:
          'Neural networks are computing systems inspired by biological neural networks. They consist of layers of interconnected nodes that process information. Deep learning uses multi-layered neural networks to solve complex problems.',
        category: 'technology',
      },
    ];

    // Add each document to the system
    for (const doc of sampleDocuments) {
      await axios.post(`${baseURL}/documents`, doc);
      console.log(`✓ Added: ${doc.title}`);
    }

    console.log('\nDocuments added successfully! Now testing semantic search...\n');

    // Test semantic searches that don't use exact keywords
    const testQueries = [
      'artificial intelligence and computers', // Should find ML and neural network docs
      'how to stay healthy while eating', // Should find cooking tips
      'working from home effectively', // Should find remote work practices
      'deep learning algorithms', // Should find both AI-related documents
    ];

    // Run each test query and display results
    for (const query of testQueries) {
      console.log(`🔍 Searching for: "${query}"`);

      const response = await axios.post(`${baseURL}/search`, {
        query,
        limit: 3,
      });

      const { results, search_time_ms } = response.data;

      console.log(`   Found ${results.length} results in ${search_time_ms}ms`);

      // Display the most relevant results
      results.forEach((doc, index) => {
        console.log(`   ${index + 1}. ${doc.title} (similarity: ${doc.similarity_score})`);
        console.log(`      Category: ${doc.category}`);
        console.log(`      Preview: ${doc.preview}\n`);
      });

      console.log('---\n');
    }
  } catch (error) {
    console.error('Test failed:', error.response?.data || error.message);
  }
}

// Start the server and run tests
console.log('Make sure your server is running with: node server.js\n');
testSemanticSearch();

Install axios for the test script and run it:

npm install axios

# Start your server in one terminal
node server.js

# Run the test in another terminal
node test-search.js

You should see output showing how the semantic search finds relevant documents even when the search terms don't appear exactly in the document text.

For example, searching for "artificial intelligence and computers" should return documents about machine learning and neural networks, demonstrating the power of semantic understanding.

The similarity scores help you understand how closely each result matches the query - scores closer to 1.0 indicate higher semantic similarity.

Performance optimization

As your document collection grows, you'll want to optimize performance.

First, let's properly configure the pgvector indexes. The default index settings work for small datasets, but you'll need to tune them for larger collections:

-- Drop existing indexes to recreate with optimal settings
DROP INDEX IF EXISTS documents_embedding_idx;

-- Create optimized indexes based on your data size
-- Rule of thumb: lists = rows / 1000, with minimum of 10
CREATE INDEX documents_embedding_idx ON documents
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);  -- Adjust based on your document count

-- Update table statistics for optimal query planning
ANALYZE documents;

These indexes dramatically improve query speed by creating approximate nearest neighbor searches instead of comparing every vector.

The lists parameter should be adjusted based on your data size, more documents need more lists for optimal performance.

Conclusion

You now should have a solid foundation for a semantic search system that uses AI embeddings alongside pgvector in Neon.

You've built a simple semantic search system that demonstrates the power of AI embeddings with Neon and pgvector.

The foundation you've built can be extended with features like real-time search suggestions or multilingual support. The principles of semantic search and vector similarity will enable you to create intelligent applications that understand user intent and context.

Additional Resources

app.build – Open-Source AI Agent That Builds Full-Stack Apps

David Gomes — Wed, 04 Jun 2025 20:41:52 +0000

Code generation has been one of the most interesting use cases for LLMs. While the best models can generate decent code for isolated problems, there is a big gap between these code snippets and fully functional applications. Agent-based solutions are better suited to create apps. To close the gap, an agent needs to iterate on the code, test the result and make decisions based on the feedback - either human comments or technical (like logs, test results, linters output etc.). In the last few months, a lot of AI app builders have come to market to tackle this problem, but the gap is not closed yet.

And today, Neon is launching app.build - an open-source agent that can build and deploy full-stack applications, with end-to-end tests and automated deployments. We want to build a community around this project, since we know a lot of developers are interested in hacking in this space, particularly by bringing their own models and doing most things locally.

How can I try it then?

npx @app.build/cli

This very simple command is all you need to get started. Notice that, by default, it will ask you to sign in with GitHub. Any application that you generate will have its own repository (in your GitHub account), and will be deployed to the Internet with a real backend and a real database. This is our "managed service" experience. We're also working on a smoother way for you to run everything locally (which will include being able to choose which models you want for which tasks, and even self-hosted models).

Can I run this locally?

Yes, everything runs locally. Our README.md contains instructions for how to run this locally, but we have yet to create a full guide for running both the CLI and the agent locally with more detail.

We also want to write down instructions on how you can bring your own models (including self-hosted ones).

What features does the platform offer?

Our CLI allows both creating new apps from scratch, and iterating on previously created apps (add new features or other types of changes). It is extremely barebones for now!

But our platform gives you:

An authentication provider (it defaults to Neon Auth)
A hosted database (it defaults to Neon Postgres)
Your app's frontend and backend get deployed immediately (in Neon and Koyeb's infrastructure)
Hosted repository on your own GitHub account

How does the agent work?

We've taken a lot of steps towards having an agent architecture that is focused on generating really high-quality applications that actually work as intended. This is why our agent writes end-to-end tests and actually runs them to succession as part of the generation pipeline.

Furthermore, we have a very solid base template for every app (for now, just 1 template, which is for web apps), and we made sure that the agent is extremely well-versed on all the technologies that are used in this template (Fastify, Drizzle, React, Vite, etc.).

Our agent follows the "divide-and-conquer" principle — app creation is decomposed into multiple tasks, and each can be solved independently. Furthermore, those tasks themselves may have their own subtasks, so we don't rely on LLMs generating large chunks of code at once. Each particular task passes a series of checks (e.g. Does it compile? Does it pass ESLint? Does it pass tests?), thus ensuring the final app works.

In future blog posts and talks, we'll definitely dive deeper into the inner workings of the agent.

How can I get involved?

For now, we're hacking over at github.com/appdotbuild. You can see all the code, all the commits and all of our planned tasks. We hope to create a Discord channel, and a proper public-facing roadmap soon!

We're also currently working on more blog posts and videos explaining our code generation pipeline, and other architectural decisions we made while building the agent.

Comparing Text Search Strategies pg_search vs. tsvector vs. External Engines

Rishi Raj Jain — Thu, 08 May 2025 17:01:53 +0000

Choosing the Right Search Approach for Your Application with PostgreSQL and Neon Postgres.

When implementing search in your application, you need to choose the right text search approach. This guide compares PostgreSQL's built-in tsvector, the pg_search extension, and external search engines to help you select the best option for your needs.

Built-in PostgreSQL `tsvector` for Text Search

PostgreSQL includes native full-text search capabilities using the tsvector data type and tsquery search expressions. The tsvector data type is a specialized data structure that PostgreSQL uses to represent documents in a form optimized for search.

This built-in functionality works for basic search needs without requiring any additional extensions. It's perfect for smaller applications or when you don't require advanced search features.

Example of Using `tsvector`

To use tsvector, you need to create a table with a column to store the search vector. You can then convert text into the tsvector format and create an index for efficient searching. This allows your queries to run faster, even as your dataset grows.

CREATE TABLE articles (
    id SERIAL PRIMARY KEY,
    title TEXT,
    content TEXT,
    search_vector tsvector
);

-- Update the search vector
UPDATE articles
SET search_vector = to_tsvector('english', title || ' ' || content);

-- Create an index on the search vector
CREATE INDEX idx_search_vector ON articles USING GIN (search_vector);

-- Query using tsquery
SELECT id, title
FROM articles
WHERE search_vector @@ to_tsquery('english', 'database & performance');

This example shows how to:

Create a table with a column to store search vectors
Convert text to the tsvector format using the to_tsvector function
Create a GIN index to speed up searches
Search for articles containing both "database" and "performance"

While tsvector meets basic search needs, it has limitations with relevance ranking, handling typos, and complex search patterns. It’s suitable for small to medium-sized datasets where advanced features aren’t necessary.

Pros of `tsvector`

Simplicity: No need for any extensions, just native PostgreSQL functionality.
Integrated with Postgres: Works seamlessly within the same database, avoiding the need for additional services.
Low overhead: Since tsvector is part of Postgres, it doesn’t require a separate system to maintain.

Cons of `tsvector`

Limited relevance ranking: It doesn't automatically rank search results based on relevance.
No typo tolerance: Exact matches are required, so if a user misspells a search term, it won’t return relevant results.
Complex queries: Handling more complex queries like fuzzy matching or phrase proximity is not possible out-of-the-box.

Extending `tsvector` with `pg_search`

The pg_search extension builds on PostgreSQL's search capabilities by adding better relevance ranking with the BM25 algorithm, fuzzy matching for handling typos, and more flexible search options.

This extension is particularly useful for applications that require more sophisticated search features without needing a separate search engine.

With pg_search, you get features such as:

Relevance Ranking: The BM25 algorithm provides an automatic ranking of search results based on how relevant they are to the query.
Fuzzy Matching: It allows for typo tolerance, meaning searches will return relevant results even when a user makes small errors.
Phrase Search: You can search for exact phrases or words that are close together, which can be especially useful in content-heavy applications.
Flexible Queries: pg_search supports more complex search expressions like partial word matches, stemming, and stop-word filtering.

Enabling `pg_search` on Neon

pg_search is currently only available on Neon projects created in an AWS region.

Adding pg_search to your Neon database is simple - just run this single SQL command:

CREATE EXTENSION IF NOT EXISTS pg_search;

That's all it takes to enable the extension. You can then use the pg_search features in your queries. The setup is straightforward, and you don’t need any additional infrastructure to get started.

Example of Using `pg_search`

After enabling the extension, you can create search indexes and run more sophisticated queries:

-- Create a BM25 index on multiple columns
CREATE INDEX article_search_idx ON articles
USING bm25 (id, title, content)
WITH (key_field='id');

-- Simple keyword search
SELECT title
FROM articles
WHERE title @@@ 'database';

-- Handling typos with fuzzy matching
SELECT title
FROM articles
WHERE id @@@ paradedb.match('title', 'database', distance => 1);

-- Sorting by relevance score
SELECT title, paradedb.score(id) AS relevance
FROM articles
WHERE content @@@ 'performance'
ORDER BY paradedb.score(id) DESC;

This code shows how to:

Create a BM25 index that covers multiple columns
Perform a basic keyword search using the @@@ operator
Find results even when the search term has typos
Sort results by relevance so the most relevant results appear first

Why Use `pg_search` on Neon?

Using pg_search on Neon gives you:

Better search capabilities: With ranking, typo tolerance, and complex query options, pg_search provides much more functionality than tsvector.
A fully managed PostgreSQL experience: You don’t need to set up or maintain a separate search service. Neon handles everything for you, from scaling to backups.
Data consistency: Since the search index is part of your PostgreSQL database, there’s no need to worry about synchronizing data between separate systems.
Simple architecture: With pg_search running on Neon, you avoid the complexity of managing an external search engine while still getting advanced search features.

pg_search is a great choice for applications that need more advanced search features but want to avoid the complexity of managing a separate search engine.

External Search Engines (e.g., Elasticsearch)

External search engines like Elasticsearch provide specialized search features for complex use cases and very large datasets. These engines are designed to scale out across many servers and handle high-performance, low-latency search queries.

While these engines offer powerful capabilities, they come with trade-offs:

You need to set up and maintain additional infrastructure: External search engines require managing separate servers or cloud services, which can increase operational overhead.
You must keep your database and search index synchronized: Ensuring that your external search engine stays in sync with your PostgreSQL database can introduce complexity, especially as your data changes.
Increased complexity: Managing another system means additional configuration, monitoring, and troubleshooting.
Higher operational costs: Running an external search engine comes with extra costs, both in terms of infrastructure and developer time.

External search engines like Elasticsearch provide powerful features such as:

Distributed search: Handles large-scale search queries across many machines.
Complex querying: Offers advanced querying capabilities such as aggregations, nested fields, and more.
Real-time indexing: Updates search results in real time as new data is indexed.

Use Cases for External Search Engines

You might choose an external search engine when:

You have extremely large datasets (billions of records) that require distributed search across multiple servers.
Your search needs include specialized features not available in PostgreSQL, such as advanced analytics, geographic search, or machine learning integration.
You have the resources to manage additional infrastructure and complexity.

Comparison Summary

Feature	`tsvector`	`pg_search` on Neon	External Engines
Setup	Built into Postgres	Easy setup on Neon	Separate system to install, configure and maintain
Relevance Ranking	Basic	BM25 ranking	Advanced ranking options
Typo Tolerance	No	Yes	Yes
Query Flexibility	Limited	Good	Extensive
Scaling	Limited by Postgres	Managed by Neon	Requires manual scaling
Cost	Included with Postgres	Included with Neon	Additional infrastructure costs
Maintenance	Minimal	Handled by Neon	Requires ongoing maintenance

Which Option Should You Choose?

With all these options available, how do you choose the right one for your application? Here are some guidelines:

Use built-in `tsvector` when:

You have simple search requirements
Your dataset is small to medium-sized
You want to use only built-in PostgreSQL features

Use `pg_search` on Neon when:

You need better search relevance and typo tolerance
You want to avoid setting up separate systems
You prefer a managed database experience
Your search needs are important but don't require specialized features

Consider external search engines when:

You have extremely large datasets (billions of records)
Your search needs include specialized features not available in PostgreSQL
You have the resources to manage additional infrastructure

For most web applications, content sites, and e-commerce platforms, pg_search on Neon provides a great balance between search features and simplicity. It extends PostgreSQL's capabilities without requiring you to manage separate systems or synchronize data.

Conclusion

When choosing a search strategy, start with the simplest option that meets your needs. For many applications, pg_search on Neon offers a great middle ground - better search features than native PostgreSQL without the complexity of a separate search system.

Selecting the right search approach allows you to provide good search functionality to your users while keeping your application architecture as simple as possible.

Coding With Cursor and Windsurf Side by Side

Sam シ — Thu, 27 Mar 2025 14:27:18 +0000

Getting a taste of vibe coding

Since the release of GitHub Copilot, AI integration in IDEs has evolved dramatically. Now, two IDEs are being talked about more than any other: Windsurf and Cursor. Both tools are forks of VSCode, offer a similar set of features, and use Anthropic’s Claude model under the hood.

I vibe coded with both IDEs and took some notes, using my experience developing the Neon Python driver as a guiding example.

What feels the same?

On the surface, Windsurf and Cursor feel strikingly similar. So much so that at first glance, you might think you’re simply looking at two VSCode windows. This means you get the familiar editor you already know and love, with the extensive extension marketplace and setting customization still available.

Both editors provide the same general interface with the chat window on the right, and the same feature set you’ve come to expect from AI-powered IDEs, like better tab completions, multi file refactoring, agent-based workflows, and context management.

Tab Completions: Cursor’s Tab vs. Windsurf’s Tabs

I find tab completions one of the most important features of any AI-powered IDE, and it’s where I noticed the biggest productivity gains when developing the Neon Python driver compared to my old setup.

While both Windsurf and Cursor offer far better autocomplete experiences than GitHub Copilot, Cursor’s tab completions are particularly impressive. Specifically, I’d often make a small change at the top of a file that would need further adjustments below, and Cursor consistently predicted exactly what I needed next—I could just tab-tab-tab my way through the file.

For a long time, Cursor held the edge in tab completions when compared to Windsurf’s SuperComplete. But Windsurf is shipping fast. As I was writing this article, they released Windsurf Wave 5, which introduced the Windsurf Tab, and the gap has narrowed. Previously, Cursor felt more aware of the codebase for tabs and provided more useful completions, but Windsurf autocompletions now benefit from the same context engine that powers its AI Agent Cascade, along with a larger and more powerful model.

Context management: Attaching the relevant information

Context management, or the way IDEs handle relevant information to inform their AI suggestions, is another important differentiator. Cursor offers robust support for custom rules that can be conditionally applied based on file type, directory, and other factors. For instance, I easily defined a set of Python rules with strict type hinting, NumPy-style docstrings, and other linting preferences. Throughout my development, Cursor adhered perfectly to these rules, saving me a lot of time in docstring writing and code-style preference refactoring. Windsurf also has built-in rule management, with support for both global rules and rules local to the project, but not for rule file pattern matching or semantic description attachment.

Yet Windsurf excels in one crucial area: context indexing. Its automatic indexing and understanding of the project structure felt superior. For example, when I made changes to the core logic of the application, Windsurf would reach out to the tests directory and update the relevant unit and integration tests. Cursor is also capable of this, but it requires prompting to get it there, which gets tedious and can easily be forgotten. On the other hand, it offers a wider variety of options for attaching content including most recent changes, git history, lint errors, and most importantly custom docs. Here, at the beginning of my project I was able to create a custom documentation context for Psycopg which I could easily reference.

Windsurf offers only a pre-set documentation library without the means to extend it, though you can attach the link of the documentation as web context to achieve a very similar result. Despite this limitation in customizable context attachments, Windsurf’s automatic context management is really a standout feature and as AI models continue to evolve, this strength will only become more pronounced.

Agent mode: Cursor’s Composer vs Windsurf’s Cascade

Both Windsurf’s Cascade and Cursor’s Composer offer agentic workflows, which allow LLMs to intelligently interact with your codebase, automate repetitive tasks, and implement entire features.

In practice, I found that both still struggled with more complex coding scenarios. For instance, when implementing Python-to-Postgres type conversions in the Neon driver, both agents had difficulty leveraging Psycopg effectively, because the conversion needed to happen without establishing an actual database connection. Even after I devised a clear implementation plan and explicitly provided it in the prompt instructions, neither IDE could successfully execute without significant manual prompting.

All hope is not lost, as both agents shined in many other tasks like tests and frontend development. In the case of testing, they generated robust, comprehensive unit and integration tests, accelerating development by at least a few hours without any detailed prompting. For this alone, the agents are well worth having integrated in the IDE. What’s more, to see how each of these fared in more line-heavy development, I tried both out in a React project of mine. After asking for the same component, they each generate working solutions, though Windsurf clearly had a broader understanding of my codebase, utilising some of my custom hooks and API endpoints many directories away.

Conclusion

Both Cursor and Windsurf significantly boosted my productivity when porting the SQL-over-HTTP driver to Python. They both have strengths in different areas —to me,

Cursor excelled in tab completions, and offered tighter, more intuitive control, providing better confidence in the AI coding workflow.
Windsurf offered better and automatic context indexing, leading to a more “magical” feeling agentic workflow.

This vibe coding experiment not only provided insights into these two IDEs but also unveiled a new way to use Neon. You can now leverage Neon more effectively in Python serverless functions (e.g., AWS Lambdas or Azure Functions) by using SQL over HTTP for efficient single-shot queries and transactions.

Check out the repository here!

Building a Multi-Step Form with Laravel Volt, Folio, and Neon Postgres

Taraneh Dohmer — Fri, 14 Feb 2025 18:36:37 +0000

In this guide, we'll walk through the process of building a multi-step form using Laravel Volt, Folio, and Neon Postgres.

Laravel Volt provides reactivity for dynamic form interactions, Folio offers file-based routing for a clean project structure, and Neon Postgres serves as our scalable database solution.

Our example app will be a job application form with multiple steps, including personal information, education, and work experience.

Prerequisites

Before we begin, make sure you have:

PHP 8.1 or higher installed
Composer for managing PHP dependencies
A Neon account for Postgres hosting
Basic familiarity with Laravel and Postgres

Setting up the Project

Let's start by creating a new Laravel project and setting up the necessary components.

Create a new Laravel project:

   composer create-project laravel/laravel job-application-form
   cd job-application-form

Install Laravel Folio for file-based routing:

   composer require laravel/folio

Install the Volt Livewire adapter for Laravel, this will also install the Livewire package:

   composer require livewire/volt

After installing Volt, you can install the Volt service provider:

   php artisan volt:install

Configuring the Database Connection

Update your .env file with your Neon Postgres credentials:

DB_CONNECTION=pgsql
DB_HOST=your-neon-hostname.neon.tech
DB_PORT=5432
DB_DATABASE=your_database_name
DB_USERNAME=your_username
DB_PASSWORD=your_password

Replace your-neon-hostname.neon.tech, your_database_name, your_username, and your_password with your Neon Postgres connection details.

Database Design

Let's create the database migrations for our job application form. We'll use separate tables for each section and leverage Postgres JSON columns for flexible data storage for additional information.

First, let's create the migration for the applicants table using the following artisan command:

php artisan make:migration create_applicants_table

Note that the create_applicants_table migration name follows the Laravel convention of create_{table_name}_table, where {table_name} is the name of the table you're creating. That way, Laravel can automatically determine the table name from the migration name, and also it will be easier to identify the purpose of the migration file by its name for other developers.

This command generates a new migration file in the database/migrations directory. Open the newly created file and update its content as follows:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up()
    {
        Schema::create('applicants', function (Blueprint $table) {
            $table->id();
            $table->string('first_name');
            $table->string('last_name');
            $table->string('email')->unique();
            $table->jsonb('additional_info')->nullable();
            $table->timestamps();
        });
    }

    public function down()
    {
        Schema::dropIfExists('applicants');
    }
};

This migration creates the applicants table with fields for first_name, last_name, and email. The email field is set as unique to prevent duplicate applications. We've also included a jsonb column called additional_info for storing any extra data that doesn't fit into the predefined columns. This flexibility is one of the advantages of using Postgres with Laravel.

Next, let's create the migration for the educations table:

php artisan make:migration create_educations_table

Update the newly created migration file with the following content:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up()
    {
        Schema::create('educations', function (Blueprint $table) {
            $table->id();
            $table->foreignId('applicant_id')->constrained()->onDelete('cascade');
            $table->string('institution');
            $table->string('degree');
            $table->date('start_date');
            $table->date('end_date')->nullable();
            $table->jsonb('additional_info')->nullable();
            $table->timestamps();
        });
    }

    public function down()
    {
        Schema::dropIfExists('educations');
    }
};

This migration creates the educations table. It includes a foreign key applicant_id that references the id column in the applicants table. The onDelete('cascade') ensures that if an applicant is deleted, their education records are also removed. We've included fields for the institution, degree, and start/end dates. Again, we have an additional_info jsonb column for flexibility.

Finally, let's create the migration for the work experiences table:

php artisan make:migration create_work_experiences_table

Update this migration file with the following content:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up()
    {
        Schema::create('work_experiences', function (Blueprint $table) {
            $table->id();
            $table->foreignId('applicant_id')->constrained()->onDelete('cascade');
            $table->string('company');
            $table->string('position');
            $table->date('start_date');
            $table->date('end_date')->nullable();
            $table->text('responsibilities');
            $table->jsonb('additional_info')->nullable();
            $table->timestamps();
        });
    }

    public function down()
    {
        Schema::dropIfExists('work_experiences');
    }
};

This migration creates the work_experiences table. Similar to the educations table, it has a foreign key relationship with the applicants table. It includes fields for the company, position, start/end dates, and responsibilities. The responsibilities field is of type text to allow for longer descriptions. We've also included an additional_info jsonb column here.

Now that we've created all our migrations, we can run them to create the tables in our database:

php artisan migrate

This command will execute all the migrations we've just created, setting up the database schema for our job application form.

One thing to note is that we've used the jsonb column type for storing additional information in each table. This allows us to store flexible data structures without needing to define a fixed schema. Postgres' JSONB data type is ideal for this use case.

For your Laravel migrations, you should not use the Neon Postgres Pooler. The Pooler is designed to manage connections for long-running processes, such as web servers, and is not necessary for short-lived processes like migrations.

Creating Models

Next, let's create models for our Applicant, Education, and WorkExperience tables. Models in Laravel are used to interact with database tables and represent the data in your application in an object-oriented way.

Laravel provides an easy way to generate models using the artisan command. To create the Applicant model run:

php artisan make:model Applicant

This command creates a new file app/Models/Applicant.php. Open this file and update it with the following content:

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;

class Applicant extends Model
{
    use HasFactory;

    protected $fillable = [
        'first_name',
        'last_name',
        'email',
        'additional_info'
    ];

    protected $casts = [
        'additional_info' => 'array',
    ];

    public function educations()
    {
        return $this->hasMany(Education::class);
    }

    public function workExperiences()
    {
        return $this->hasMany(WorkExperience::class);
    }
}

Now, create the Education model:

php artisan make:model Education

Update the newly created file at app/Models/Education.php with the following content:

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;

class Education extends Model
{
    use HasFactory;

    public $table = 'educations';

    protected $fillable = [
        'applicant_id',
        'institution',
        'degree',
        'start_date',
        'end_date',
        'additional_info'
    ];

    protected $casts = [
        'start_date' => 'date',
        'end_date' => 'date',
        'additional_info' => 'array',
    ];

    public function applicant()
    {
        return $this->belongsTo(Applicant::class);
    }
}

Finally, create the WorkExperience model:

php artisan make:model WorkExperience

And update the app/Models/WorkExperience.php file with the following content:

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;

class WorkExperience extends Model
{
    use HasFactory;

    protected $fillable = [
        'applicant_id',
        'company',
        'position',
        'start_date',
        'end_date',
        'responsibilities',
        'additional_info'
    ];

    protected $casts = [
        'start_date' => 'date',
        'end_date' => 'date',
        'additional_info' => 'array',
    ];

    public function applicant()
    {
        return $this->belongsTo(Applicant::class);
    }
}

Let's quickly note down the most important parts in these model definitions:

We've used the $fillable property to specify which attributes can be mass-assigned. This is a security feature to prevent unintended mass assignment vulnerabilities.
We've defined relationships between models. An Applicant has many Education and WorkExperience records, while Education and WorkExperience belong to an Applicant.
We've used the $casts property to automatically cast certain attributes to specific types. For example, we're casting the additional_info field to an array, which works well with Postgres' JSONB column type.
The start_date and end_date fields are cast to date objects, which allows for easy date manipulation in PHP.

These models will allow us to easily interact with our database tables using Laravel's Eloquent ORM. They provide a convenient way to retrieve, create, update, and delete records, as well as define relationships between different tables.

Creating a layout for the multi-step form

Before we create the form components, let's set up a layout for our multi-step form. We'll create a main layout file that includes the necessary CSS and JavaScript assets including the Livewire scripts.

Create a new Blade layout file at resources/views/layouts/app.blade.php:

<!DOCTYPE html>
<html lang="{{ str_replace('_', '-', app()->getLocale()) }}">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Job Application Form</title>
    <script src="https://cdn.tailwindcss.com"></script>
    @vite(['resources/css/app.css', 'resources/js/app.js'])
    @livewireStyles
</head>
<body>
    <div class="container mx-auto mt-8">
        @yield('content')
    </div>

    @livewireScripts
</body>
</html>

In this layout file:

We've included the necessary meta tags for character encoding, viewport settings and the page title.
We've used the @vite directive to include the CSS and JavaScript assets. This directive is provided by the Laravel Vite package, which integrates Laravel with the Vite build tool for modern frontend development.
We've included the Livewire styles and scripts. Livewire is a full-stack framework for Laravel that allows you to build dynamic interfaces without writing JavaScript.

To compile the frontend assets, you'll need to run the following commands:

npm install
npm run build

Implementing File-based Routing with Folio

Laravel Folio was introduced in 2023, and it offers a new approach to routing in Laravel applications.

It simplifies routing by allowing you to create routes simply by adding Blade templates to a specific directory. This file-based routing system makes your project structure cleaner and more intuitive.

It is not a replacement for Laravel's built-in routing system but rather a complementary feature that simplifies routing for certain types of applications.

First, let's set up the directory structure for our multi-step form. Create the following directory structure in your resources/views/pages folder:

resources/
└── views/
    └── pages/
        ├── index.blade.php
        └── apply/
            ├── index.blade.php
            ├── personal-info.blade.php
            ├── education.blade.php
            ├── work-experience.blade.php
            └── review.blade.php
            └── confirmation.blade.php

With Folio, each of these Blade files automatically becomes a route. For example:

pages/index.blade.php will be accessible at the root URL /
pages/apply/personal-info.blade.php will be accessible at /apply/personal-info

To create a Folio page, you can use the php artisan folio:page command. For example, to create a page for the personal information step:

php artisan folio:page apply/personal-info

The above will create a blade file for the in resources/views/pages/apply/personal-info.blade.php:

<div>
    <h2>Personal Information</h2>
    <!-- Your form content will go here -->
</div>

You can list all available Folio routes using the following Artisan command:

php artisan folio:list

You can create similar pages for the education, work experience, and review steps:

php artisan folio:page apply/education
php artisan folio:page apply/work-experience
php artisan folio:page apply/review

We will update these files with the form components later in the guide.

The main thing to remember here is that with Folio, you don't need to manually define routes in a separate routes file. The mere presence of a Blade file in the pages directory automatically creates a corresponding route.

Building the Multi-Step Form with Volt

Volt is a powerful addition to Laravel Livewire that allows you to build reactive components without writing JavaScript. Unlike traditional Livewire components, Volt lets you define your component's state and validation rules directly in the view file, eliminating the need for a separate component class.

Let's create Volt components for each step of our multi-step form.

Personal Information Form

First, create the personal information form component:

php artisan make:volt personal-info-form

That will create a file at resources/views/livewire/personal-info-form.blade.php. Update the file with the following content:

<?php

use function Livewire\Volt\state;
use function Livewire\Volt\rules;

state([
    'first_name' => '',
    'last_name' => '',
    'email' => '',
]);

rules([
    'first_name' => 'required|min:2',
    'last_name' => 'required|min:2',
    'email' => 'required|email|unique:applicants,email',
]);

$saveAndContinue = function () {
    $this->validate();

    $applicant = \App\Models\Applicant::create($this->only(['first_name', 'last_name', 'email']));
    session(['applicant_id' => $applicant->id]);

    return redirect()->route('apply.education');
};

?>

<div class="max-w-lg p-8 mx-auto bg-white rounded-lg shadow-md">
    <h2 class="mb-6 text-2xl font-semibold text-gray-800">Personal Information</h2>

    <form wire:submit.prevent="saveAndContinue">
        <!-- First Name -->
        <div class="mb-4">
            <label for="first_name" class="block text-sm font-medium text-gray-700">First Name</label>
            <input type="text" id="first_name" wire:model="first_name" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('first_name')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-4">
            <label for="last_name" class="block text-sm font-medium text-gray-700">Last Name</label>
            <input type="text" id="last_name" wire:model="last_name" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('last_name')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-6">
            <label for="email" class="block text-sm font-medium text-gray-700">Email</label>
            <input type="email" id="email" wire:model="email" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('email')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div>
            <button type="submit" class="w-full px-4 py-2 text-white transition duration-200 bg-indigo-600 rounded-md hover:bg-indigo-700">
                Next
            </button>
        </div>
    </form>
</div>

Quick explanation of the code above:

We define the component's state using the state function, which initializes the form fields.
The rules function sets up validation rules for each field.
The saveAndContinue function handles form submission. It validates the form, creates a new Applicant record, stores the applicant_id in the session, and redirects to the next step.
The form fields are bound to the component's state using wire:model.
Validation errors are displayed using @error.

In the same way, you can create components for the education, work experience, and review steps.

Education Form

Next, create the education form component:

php artisan make:volt education-form

Update resources/views/livewire/education-form.blade.php:

<?php

use function Livewire\Volt\state;
use function Livewire\Volt\rules;

state([
    'institution' => '',
    'degree' => '',
    'start_date' => '',
    'end_date' => '',
]);

rules([
    'institution' => 'required|min:2',
    'degree' => 'required|min:2',
    'start_date' => 'required|date',
    'end_date' => 'nullable|date|after:start_date',
]);

$saveAndContinue = function () {
    $this->validate();

    $applicantId = session('applicant_id');
    \App\Models\Education::create(array_merge($this->all(), ['applicant_id' => $applicantId]));

    return redirect()->route('apply.work-experience');
};

?>

<div class="max-w-lg p-8 mx-auto bg-white rounded-lg shadow-md">
    <h2 class="mb-6 text-2xl font-semibold text-gray-800">Education</h2>

    <form wire:submit.prevent="saveAndContinue">
        <div class="mb-4">
            <label for="institution" class="block text-sm font-medium text-gray-700">Institution</label>
            <input type="text" id="institution" wire:model="institution" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('institution')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-4">
            <label for="degree" class="block text-sm font-medium text-gray-700">Degree</label>
            <input type="text" id="degree" wire:model="degree" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('degree')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-4">
            <label for="start_date" class="block text-sm font-medium text-gray-700">Start Date</label>
            <input type="date" id="start_date" wire:model="start_date" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('start_date')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-6">
            <label for="end_date" class="block text-sm font-medium text-gray-700">End Date</label>
            <input type="date" id="end_date" wire:model="end_date" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('end_date')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div>
            <button type="submit" class="w-full px-4 py-2 text-white transition duration-200 bg-indigo-600 rounded-md hover:bg-indigo-700">
                Next
            </button>
        </div>
    </form>
</div>

Work Experience Form

Next, let's create the work experience form component:

php artisan make:volt work-experience-form

Update resources/views/livewire/work-experience-form.blade.php similar to the previous components:

<?php

use function Livewire\Volt\state;
use function Livewire\Volt\rules;

state([
    'company' => '',
    'position' => '',
    'start_date' => '',
    'end_date' => '',
    'responsibilities' => '',
]);

rules([
    'company' => 'required|min:2',
    'position' => 'required|min:2',
    'start_date' => 'required|date',
    'end_date' => 'nullable|date|after:start_date',
    'responsibilities' => 'required|min:10',
]);

$saveAndContinue = function () {
    $this->validate();

    $applicantId = session('applicant_id');
    \App\Models\WorkExperience::create(array_merge($this->all(), ['applicant_id' => $applicantId]));

    return redirect()->route('apply.review');
};

?>

<div class="max-w-lg p-8 mx-auto bg-white rounded-lg shadow-md">
    <h2 class="mb-6 text-2xl font-semibold text-gray-800">Work Experience</h2>

    <form wire:submit.prevent="saveAndContinue">
        <div class="mb-4">
            <label for="company" class="block text-sm font-medium text-gray-700">Company</label>
            <input type="text" id="company" wire:model="company" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('company')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-4">
            <label for="position" class="block text-sm font-medium text-gray-700">Position</label>
            <input type="text" id="position" wire:model="position" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('position')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-4">
            <label for="start_date" class="block text-sm font-medium text-gray-700">Start Date</label>
            <input type="date" id="start_date" wire:model="start_date" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('start_date')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-4">
            <label for="end_date" class="block text-sm font-medium text-gray-700">End Date</label>
            <input type="date" id="end_date" wire:model="end_date" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
            @error('end_date')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div class="mb-6">
            <label for="responsibilities" class="block text-sm font-medium text-gray-700">Responsibilities</label>
            <textarea id="responsibilities" wire:model="responsibilities" class="block w-full px-3 py-2 mt-1 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"></textarea>
            @error('responsibilities')
                <span class="text-sm text-red-500">{{ $message }}</span>
            @enderror
        </div>

        <div>
            <button type="submit" class="w-full px-4 py-2 text-white transition duration-200 bg-indigo-600 rounded-md hover:bg-indigo-700">
                Next
            </button>
        </div>
    </form>
</div>

Review Form

Finally, create the review form component:

php artisan make:volt review-form

Update resources/views/livewire/review-form.blade.php as we did for the other components:

<?php

use function Livewire\Volt\state;
use function Livewire\Volt\{mount};
use App\Models\Applicant;

state(['applicant' => null]);

mount(function () {
    $applicantId = session('applicant_id');
    $this->applicant = Applicant::with(['educations', 'workExperiences'])->findOrFail($applicantId);
});

$submit = function () {
    session()->flash('message', 'Your application has been submitted successfully!');
    return redirect()->route('apply.confirmation');
};

?>

<div class="max-w-2xl p-8 mx-auto bg-white rounded-lg shadow-md">
    <h2 class="mb-6 text-2xl font-semibold text-gray-800">Review Your Application</h2>

    <div class="mb-6">
        <h3 class="mb-3 text-xl font-semibold text-gray-700">Personal Information</h3>
        <p><strong>Name:</strong> {{ $applicant->first_name }} {{ $applicant->last_name }}</p>
        <p><strong>Email:</strong> {{ $applicant->email }}</p>
    </div>

    <div class="mb-6">
        <h3 class="mb-3 text-xl font-semibold text-gray-700">Education</h3>
        @foreach($applicant->educations as $education)
            <div class="p-4 mb-4 rounded-lg shadow-sm bg-gray-50">
                <p><strong>Institution:</strong> {{ $education->institution }}</p>
                <p><strong>Degree:</strong> {{ $education->degree }}</p>
                <p><strong>Period:</strong> {{ $education->start_date }} - {{ $education->end_date ?: 'Present' }}</p>
            </div>
        @endforeach
    </div>

    <div class="mb-6">
        <h3 class="mb-3 text-xl font-semibold text-gray-700">Work Experience</h3>
        @foreach($applicant->workExperiences as $experience)
            <div class="p-4 mb-4 rounded-lg shadow-sm bg-gray-50">
                <p><strong>Company:</strong> {{ $experience->company }}</p>
                <p><strong>Position:</strong> {{ $experience->position }}</p>
                <p><strong>Period:</strong> {{ $experience->start_date }} - {{ $experience->end_date ?: 'Present' }}</p>
                <p><strong>Responsibilities:</strong> {{ $experience->responsibilities }}</p>
            </div>
        @endforeach
    </div>

    <div>
        <button wire:click="submit" class="w-full px-4 py-2 text-white transition duration-200 bg-indigo-600 rounded-md hover:bg-indigo-700">
            Submit Application
        </button>
    </div>
</div>

These Volt components handle the state management, validation, and submission logic for each step of the multi-step form. That way Volt simplifies the process of creating interactive components by allowing you to define both the logic and the template in a single file.

To use these components in your Folio pages and make the routes named, you can include them like this. Named routes allow you to easily reference routes by name throughout your application. We also need to extend a layout for each page to ensure a consistent structure.

First, in each file, you will define a named route using the name function and extend the layout.

For the resources/views/pages/apply/personal-info.blade.php file:

  <?php
  use function Laravel\Folio\name;

  name('apply.personal-info');
  ?>

  @extends('layouts.app')

  @section('title', 'Personal Information')

  @section('content')
  <div>
      <livewire:personal-info-form />
  </div>
  @endsection

We need to do the same for the other pages:

For the resources/views/pages/apply/education.blade.php file:

  <?php
  use function Laravel\Folio\name;

  name('apply.education');
  ?>

  @extends('layouts.app')

  @section('title', 'Education')

  @section('content')
  <div>
      <livewire:education-form />
  </div>
  @endsection

For the resources/views/pages/apply/work-experience.blade.php file:

  <?php
  use function Laravel\Folio\name;

  name('apply.work-experience');
  ?>

  @extends('layouts.app')

  @section('title', 'Work Experience')

  @section('content')
  <div>
      <livewire:work-experience-form />
  </div>
  @endsection

And for the resources/views/pages/apply/review.blade.php file:

  <?php
  use function Laravel\Folio\name;

  name('apply.review');
  ?>

  @extends('layouts.app')

  @section('title', 'Review')

  @section('content')
  <div>
      <livewire:review-form />
  </div>
  @endsection

Confirmation Page

Finally, create a confirmation page for the application submission:

php artisan folio:page apply/confirmation

Update the resources/views/pages/apply/confirmation.blade.php file:

<?php
use function Laravel\Folio\name;

name('apply.confirmation');
?>

@extends('layouts.app')

@section('title', 'Confirmation')

@section('content')
<div class="max-w-md p-8 mx-auto text-center bg-white rounded-lg shadow-md">
    <h2 class="mb-4 text-2xl font-semibold text-green-600">Application Submitted</h2>

    <p class="mb-6 text-gray-700">{{ session('message') }}</p>

    <a href="/" class="inline-block px-4 py-2 text-white transition duration-200 bg-indigo-600 rounded-md hover:bg-indigo-700">
        Return to Homepage
    </a>
</div>
@endsection

This page displays a success message after the application is submitted and provides a link to return to the homepage.

Testing the Multi-Step Form

To manually verify that everything works as expected, follow these steps:

If you haven't already, start the Laravel development server:

   php artisan serve

Open your browser and navigate to http://localhost:8000/apply/personal-info.
Fill out the personal information form and submit it. You should be redirected to the education form.
Fill out the education form and submit it. You should be redirected to the work experience form.
Fill out the work experience form and submit it. You should be redirected to the review page.
On the review page, verify that all the information you entered is displayed correctly.
Submit the application and verify that you see a success message.
To check if the data was persisted correctly:

Open a database client (like pgAdmin for Postgres) and connect to your Neon database.
Check the applicants, educations, and work_experiences tables. You should see your submitted data.
Verify that the applicant_id in the educations and work_experiences tables matches the id in the applicants table for your submission.

Try refreshing the page or closing and reopening your browser, then navigate back to http://localhost:8000/apply/review. You should still see your submitted data, demonstrating that the data persists across sessions.

Testing

Besides manual testing, you can also write automated tests to make sure your multi-step form works correctly. Laravel provides a testing suite that allows you to write unit, feature, and browser tests.

Create feature tests for your multi-step form to ensure each step works correctly. Here's an example for the personal info step:

<?php

namespace Tests\Feature;

use Tests\TestCase;
use App\Models\Applicant;
use Livewire\Volt\Volt;
use Illuminate\Foundation\Testing\RefreshDatabase;

class PersonalInfoTest extends TestCase
{
    use RefreshDatabase;

    public function test_can_submit_personal_info()
    {
        Volt::test('personal-info-form')
            ->set('first_name', 'John')
            ->set('last_name', 'Doe')
            ->set('email', 'john@example.com')
            ->call('saveAndContinue')
            ->assertRedirect('/apply/education');

        $this->assertDatabaseHas('applicants', [
            'first_name' => 'John',
            'last_name' => 'Doe',
            'email' => 'john@example.com',
        ]);

        $this->assertNotNull(session('applicant_id'));
    }
}

This test checks if:

The form can be submitted with valid data.
The data is correctly stored in the database.
The applicant_id is stored in the session.
The user is redirected to the next step after submission.

You can create similar tests for the education and work experience steps.

To learn more about testing in Laravel, check out the Testing Laravel Applications with Neon's Database Branching guide.

Conclusion

In this guide, we've built a multi-step form using Laravel Volt, Folio, and Neon Postgres. We've covered form validation, data storage, and routing, demonstrating how these tools can be used together to create a dynamic and interactive form.

To further improve this project, consider adding features like:

File uploads for resumes
Email notifications to applicants
An admin interface to review applications

One thing to keep in mind is always to validate and sanitize user inputs, optimize your database queries, and thoroughly test your application before deploying to production.

Additional Resources

Building a RESTful API with ASP.NET Core, Swagger, and Neon

Rishi Raj Jain — Thu, 23 Jan 2025 19:19:14 +0000

In this guide, we'll walk through the process of developing a RESTful API using ASP.NET Core, connecting it to a Neon Postgres database. We will cover CRUD operations using Entity Framework Core (EF Core), generate interactive API documentation with Swagger, and explore best practices for testing your API endpoints. As a bonus, we'll also implement JWT authentication to secure your endpoints.

Prerequisites

Before we start, make sure you have the following:

.NET SDK 8.0
Neon account for setting up your Postgres database
Postman for API testing
Basic knowledge of C# and ASP.NET Core
Familiarity with Entity Framework Core

Setting Up Your ASP.NET Core Project

First, create a new ASP.NET Core Web API project:

dotnet new webapi -n NeonApi
cd NeonApi

Install the required NuGet packages using the dotnet add package command:

dotnet add package Microsoft.EntityFrameworkCore
dotnet add package Npgsql.EntityFrameworkCore.PostgreSQL
dotnet add package Swashbuckle.AspNetCore
dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer
dotnet add package Microsoft.EntityFrameworkCore.Design

The above packages include:

Microsoft.EntityFrameworkCore - Entity Framework Core for database operations
Npgsql.EntityFrameworkCore.PostgreSQL - PostgreSQL provider for EF Core
Swashbuckle.AspNetCore - Swagger for API documentation
Microsoft.AspNetCore.Authentication.JwtBearer - JWT authentication for securing endpoints
Microsoft.EntityFrameworkCore.Design - EF Core design tools for migrations

Configuring the Neon Database

Head over to your Neon Dashboard and create a new project.

Once done, grab your database connection string and add it to your appsettings.json:

"ConnectionStrings": {
  "NeonDb": "Host=<your-host>;Database=<your-database>;Username=<your-username>;Password=<your-password>;Port=5432"
}

While you're in appsettings.json, add a section for JWT authentication after the connection string:

"Jwt": {
"SecretKey": "your-very-secure-secret-key"
}

We will cover JWT authentication in more detail later in this guide, but for now, let's focus on setting up the API.

Next, update your Program.cs file to include the database context, Swagger, and JWT authentication:

using NeonApi.Data;
using Microsoft.AspNetCore.Authentication.JwtBearer;
using Microsoft.EntityFrameworkCore;
using Microsoft.IdentityModel.Tokens;
using System.Text;

var builder = WebApplication.CreateBuilder(args);
var configuration = builder.Configuration;

// Add the Neon database context using Npgsql provider
builder.Services.AddDbContext<NeonDbContext>(options =>
    options.UseNpgsql(configuration.GetConnectionString("NeonDb")));

// Register controllers for handling incoming HTTP requests
builder.Services.AddControllers();

// Enable API endpoint exploration and Swagger documentation
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

// Set up JWT authentication to secure API endpoints
builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.TokenValidationParameters = new TokenValidationParameters
        {
            // Validate that the token is signed with the specified key
            ValidateIssuerSigningKey = true,
            IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(configuration["Jwt:SecretKey"])),

            // Disable issuer and audience validation for testing purposes
            ValidateIssuer = false,
            ValidateAudience = false
        };
    });

var app = builder.Build();

// Configure the middleware pipeline
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();
app.UseAuthentication();
app.UseAuthorization();
app.MapControllers();

app.Run();

In the above, we configure the necessary services directly within Program.cs to connect our ASP.NET Core API to Neon and secure it with JWT authentication:

We use AddDbContext to set up NeonDbContext with the Npgsql provider, connecting to the Neon database using the connection string defined in appsettings.json. Make sure to update "NeonDb" with your actual connection string key if it's named differently.
We register controllers with AddControllers(), which allows the application to handle incoming API requests and map them to their respective endpoints.
By adding EndpointsApiExplorer and SwaggerGen, we enable automatic generation of API documentation. This provides a user-friendly interface to interact with your API endpoints, accessible at /swagger.
For the JWT authentication setup, we are implementing the following:
- Here, we set up JWT authentication to protect your API routes. We use AddJwtBearer to validate tokens sent by clients.
- The TokenValidationParameters section allows us to make sure that the token is signed with the specified key. Replace "your-secret-key" in appsettings.json with a secure key unique to your application.
- For simplicity, ValidateIssuer and ValidateAudience are set to false, which means the API won't check who issued the token or its intended audience. This is useful for local development but should be tightened for production environments.

To avoid hardcoding sensitive information like the secret key, consider using environment variables or a configuration management system to securely store secrets.

Creating the Entity Framework Core Models

Data models define the structure of your database tables and the relationships between them. Here, we'll create a simple Product model to represent products in our Neon database.

In the Models folder, create a Product.cs file:

namespace NeonApi.Models
{
    public class Product
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public decimal Price { get; set; }
        public string Description { get; set; }
    }
}

Here, we define a simple Product model with four properties:

Id: This is the primary key, which will auto-increment.
Name: Stores the product's name.
Price: Holds the product's price as a decimal value.
Description: Provides additional details about the product.

Each property corresponds to a column in the database table that Entity Framework will generate for us.

Creating the Database Context

Next, we need to create a database context class, which serves as a bridge between our C# code and the Neon database.

Create a new folder named Data and add a NeonDbContext.cs file:

namespace NeonApi.Data
{
    public class NeonDbContext : DbContext
    {
        public NeonDbContext(DbContextOptions<NeonDbContext> options) : base(options) { }

        // This DbSet represents the Products table in the Neon database
        public DbSet<Product> Products { get; set; }
    }
}

The above code snippet does the following:

The NeonDbContext class inherits from DbContext, which is part of Entity Framework Core.
We pass DbContextOptions to the constructor to configure the connection to our Neon database.
The DbSet<Product> property represents the Products table. This allows us to perform CRUD operations on the Product model directly through this context.

Running Migrations to Create the Database Schema

Now that we have defined our model and context, let's generate the database schema using migrations. Open your terminal and run the following commands:

dotnet ef migrations add InitialCreate

The above command generates a migration file based on the changes made to the database schema. The migration file contains instructions to create the Products table.

Next, apply the migration to your Neon database:

dotnet ef database update

The dotnet ef database update command applies the migration to your Neon database, creating the Products table and any other necessary schema changes.

Note: Make sure your database connection string in appsettings.json is correctly configured before running the migrations. That way the changes are applied to your Neon database instance.

At this point, your database is set up and ready to store product data!

Building the API Endpoints

With the database schema in place, let's create the API endpoints to perform CRUD operations on the Products table.

In the Controllers folder, create a ProductsController.cs file with the following content:

using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.EntityFrameworkCore;
using NeonApi.Data;
using NeonApi.Models;

[Route("api/[controller]")]
[ApiController]
public class ProductsController : ControllerBase
{
    private readonly NeonDbContext _context;

    public ProductsController(NeonDbContext context)
    {
        _context = context;
    }

    [HttpGet]
    public async Task<ActionResult<IEnumerable<Product>>> GetProducts()
    {
        // Retrieve all products from the database
        return await _context.Products.ToListAsync();
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<Product>> GetProduct(int id)
    {
        // Retrieve a single product by ID
        var product = await _context.Products.FindAsync(id);
        if (product == null) return NotFound(); // Return 404 if not found
        return product;
    }

    [HttpPost]
    public async Task<ActionResult<Product>> CreateProduct(Product product)
    {
        // Add a new product to the database
        _context.Products.Add(product);
        await _context.SaveChangesAsync();

        // Return 201 Created status with the newly created product
        return CreatedAtAction(nameof(GetProduct), new { id = product.Id }, product);
    }

    [HttpPut("{id}")]
    public async Task<IActionResult> UpdateProduct(int id, Product product)
    {
        // Ensure the ID in the URL matches the ID of the provided product
        if (id != product.Id) return BadRequest();

        // Mark the product as modified
        _context.Entry(product).State = EntityState.Modified;
        await _context.SaveChangesAsync();

        // Return 204 No Content status after a successful update
        return NoContent();
    }

    [HttpDelete("{id}")]
    public async Task<IActionResult> DeleteProduct(int id)
    {
        // Find the product by ID
        var product = await _context.Products.FindAsync(id);
        if (product == null) return NotFound(); // Return 404 if not found

        // Remove the product from the database
        _context.Products.Remove(product);
        await _context.SaveChangesAsync();

        // Return 204 No Content status after successful deletion
        return NoContent();
    }
}

In the code above, we define a ProductsController to handle all CRUD operations for our Product model. Here's a breakdown of how each endpoint works:

The GetProducts method handles GET /api/products requests, fetching all products stored in the Neon database.
The GetProduct method handles GET /api/products/{id} requests to retrieve a single product by its unique ID. If no product with the given ID is found, it responds with a 404 Not Found. This ensures the client is notified when attempting to access a non-existent product.
The CreateProduct method handles POST /api/products requests to add a new product. The response uses CreatedAtAction to include a link to the newly created resource, following REST best practices.
The UpdateProduct method handles PUT /api/products/{id} requests to modify an existing product. Once the update is successful, it responds with a 204 No Content, indicating the operation was successful without returning any additional data.
The DeleteProduct method handles DELETE /api/products/{id} requests to remove a product by its ID. If the product doesn't exist, it returns a 404 Not Found response.

Each endpoint is fully asynchronous and interacts with the Neon database through the NeonDbContext context.

Setting Up Swagger for API Documentation

To document our API and provide an interactive interface for testing, we'll integrate Swagger into our ASP.NET Core project.

Swagger automatically generates OpenAPI documentation, making it easy to explore your API and test its endpoints directly from your browser.

Enabling Swagger in `Program.cs`

To set up Swagger, add the following code in the Configure method of your Program.cs file:

app.UseSwagger();
app.UseSwaggerUI(c =>
{
    // Configure Swagger UI at the root URL
    c.SwaggerEndpoint("/swagger/v1/swagger.json", "Neon API V1");
    c.RoutePrefix = string.Empty;
});

The UseSwagger middleware generates the OpenAPI documentation for your API, while UseSwaggerUI sets up the Swagger interface for interacting with your endpoints.

Running Your Application

Now that Swagger is set up, start your application using:

dotnet run

Once the application is running, you will see the port where your API is hosted (usually https://localhost:5229), eg.:

Building...
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: http://localhost:5229

Visit https://localhost:5229/swagger in your browser to access the Swagger UI.

The Swagger UI will appear, displaying all your API endpoints with detailed documentation. You can test the endpoints by sending requests directly from the UI and viewing the responses, making it easy to verify that everything works as expected.

Testing Your API with Postman

Now that your API is running, you can use Postman or any other API testing tool to interact with your endpoints. You can even use the Swagger UI to test the endpoints, but Postman provides a more robust environment for testing complex scenarios.

In this section, we'll walk through testing the CRUD operations using Postman but feel free to use any tool you're comfortable with like Insomnia or cURL.

Download and launch Postman. Create a new request to interact with your API.

Open Postman and create the following requests:

GET: /api/products

Description: Fetches all products.
Set to GET, enter https://localhost:5001/api/products, and click Send.
You should receive a 200 OK response with a list of products.

POST: /api/products

Description: Creates a new product.
Set to POST, enter https://localhost:5001/api/products, and go to Body → raw → JSON.

Add:

 {
   "name": "New Product",
   "price": 19.99,
   "description": "Sample product"
 }

Click Send. Expect a 201 Created response.

PUT: /api/products/{id}

Description: Updates a product.
Set to PUT, enter https://localhost:5001/api/products/1.

Add:

 {
   "id": 1,
   "name": "Updated Product",
   "price": 29.99,
   "description": "Updated description"
 }

Click Send. You should receive a 204 No Content.

DELETE: /api/products/{id}
- Description: Deletes a product.
- Set to DELETE, enter https://localhost:5001/api/products/1, and click Send.
- Expect a 204 No Content.

After testing, check that all changes are reflected in your Neon database. Use both Postman and Swagger UI to confirm the endpoints are functioning correctly.

Securing Your API with JWT Authentication (Bonus)

To protect your API endpoints, we’ll use JWT (JSON Web Token) authentication. By adding the [Authorize] attribute to specific controller actions, you can ensure that only authenticated users have access. Here’s how to secure the GetProducts endpoint:

[Authorize]
[HttpGet]
public async Task<ActionResult<IEnumerable<Product>>> GetProducts()
{
    return await _context.Products.ToListAsync();
}

Now, any requests to GetProducts will require a valid JWT token, if you try to access the endpoint without a token, you will receive a 401 Unauthorized response.

Generating and Using JWT Tokens

When a user successfully logs in, the server generates a JWT token containing the user's authentication details. This token typically includes claims such as the user's ID, email, and a unique identifier. The token is then signed with a secret key to ensure its integrity and prevent tampering. For example:

public string GenerateJwtToken(User user)
{
    var claims = new[]
    {
        new Claim(JwtRegisteredClaimNames.Sub, user.Id.ToString()),
        new Claim(JwtRegisteredClaimNames.Email, user.Email),
        new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString())
    };

    var key = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(configuration["Jwt:SecretKey"]));
    var creds = new SigningCredentials(key, SecurityAlgorithms.HmacSha256);

    var token = new JwtSecurityToken(
        issuer: null,
        audience: null,
        claims: claims,
        expires: DateTime.UtcNow.AddHours(1),
        signingCredentials: creds);

    return new JwtSecurityTokenHandler().WriteToken(token);
}

The token looks like this:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwiZW1haWwiOiJhZG1pbkBleGFtcGxlLmNvbSIsImp0aSI6IjMzYjQwMzIwLWUwZjItNDIwZi1iZjIwLWUwZjIwZmJmMjAwMCIsImV4cCI6MTY0MzUwNzQwMH0.7

If you were to go to jwt.io, you could paste the token and see its decoded contents.

Once the token is generated, the client stores it in local storage or session storage and includes it in the Authorization header for all subsequent requests to secured endpoints. For instance:

Authorization: Bearer <your-jwt-token>

With this header in place, the server can authenticate the user without requiring them to log in again for each request.

Conclusion

In this guide, we covered the process of building a RESTful API with ASP.NET Core, connecting it to a Neon Postgres database, and securing it with JWT authentication. We explored CRUD operations using Entity Framework Core, generated interactive API documentation with Swagger, and tested our endpoints using Postman.

As a next step, consider expanding your API with additional features, such as pagination, filtering, or sorting. You can also explore adding testing frameworks like xUnit or NUnit to write unit tests for your API endpoints.

For more information, check out:

Creating a Multi-Tenant Application with Laravel and Neon

Rishi Raj Jain — Fri, 10 Jan 2025 19:27:38 +0000

Multi-tenancy is a software architecture where a single instance of an application serves multiple tenants or clients.

Each tenant's data is isolated and remains invisible to other tenants. This approach is commonly used in Software as a Service (SaaS) applications. In this tutorial, we'll build the foundation for a multi-tenant SaaS application using Laravel and Neon.

By the end of this tutorial, you'll have a fully functional multi-tenant SaaS application where tenants can manage their own books, users, and settings, all while maintaining data isolation between tenants.

Prerequisites

Before we start, make sure you have the following:

PHP 8.1 or higher installed on your system
Composer for managing PHP dependencies
Node.js and npm for managing front-end assets
A Neon account for database hosting
Basic knowledge of Laravel and Livewire

Setting up the Project

Let's start by creating a new Laravel project and setting up the necessary components.

Creating a New Laravel Project

Open your terminal and run the following command to create a new Laravel project:

composer create-project laravel/laravel laravel-multi-tenant-saas
cd laravel-multi-tenant-saas

Installing Required Packages

For our multi-tenant SaaS application, we'll use the following package:

stancl/tenancy: A flexible multi-tenancy package for Laravel
Laravel Breeze: A minimal authentication starter kit for Laravel

Start by installing the stancl/tenancy package:

composer require stancl/tenancy

After installing the package, let's set up the tenancy:

php artisan tenancy:install

return [
    // ...
    App\Providers\TenancyServiceProvider::class,
],

Let's install Laravel Breeze with the Blade views:

composer require laravel/breeze --dev
php artisan breeze:install blade

Next, install the required NPM packages:

npm install
npm run dev

Setting up the Database

Update your .env file with your Neon database credentials:

DB_CONNECTION=pgsql
DB_HOST=your-neon-hostname.neon.tech
DB_PORT=5432
DB_DATABASE=your_database_name
DB_USERNAME=your_username
DB_PASSWORD=your_password

After updating the .env file, run the database migrations:

php artisan migrate

Implementing Multi-Tenancy

Now that we have our basic setup, let's implement multi-tenancy in our application.

Creating the Tenant Model

Create a Tenant model:

php artisan make:model Tenant

Update the app/Models/Tenant.php file:

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Stancl\Tenancy\Database\Models\Tenant as BaseTenant;
use Stancl\Tenancy\Contracts\TenantWithDatabase;
use Stancl\Tenancy\Database\Concerns\HasDatabase;
use Stancl\Tenancy\Database\Concerns\HasDomains;

class Tenant extends BaseTenant implements TenantWithDatabase
{
    use HasFactory, HasDatabase, HasDomains;

}

This model extends the base Tenant model provided by the tenancy package and implements the TenantWithDatabase interface. We've also defined the fillable attributes and custom columns for our tenant.

The HasDatabase and HasDomains traits provided by the tenancy package allow us to manage tenant-specific databases and domains. This essentially means that each tenant will have its own database and domain providing data isolation between tenants.

To learn more about the tenancy package event system and how to customize the tenant model, refer to the stancl/tenancy documentation.

Configuring Tenancy

Update the config/tenancy.php file to use our custom Tenant model:

'tenant_model' => \App\Models\Tenant::class,

Also, update the central domains configuration:

'central_domains' => [
    'laravel-multi-tenant-saas.test',
    'localhost',
    'example.com',
],

Replace the default central domains with your own domain names.

This is an important part as this is how the tenancy package will determine which domain belongs to which tenant and load the tenant-specific data accordingly.

Feel free to review the other configuration options in the config/tenancy.php file to customize the tenancy behavior based on your requirements.

Creating Tenant Migrations

The tenancy package has built-in event listeners that automatically run tenant-specific migrations when a tenant is created. For this we need to make sure that all of the tenant-specific migrations are in the database/migrations/tenant directory.

As each tenant will have its own database, the migrations in the tenant directory will be used to create tenant-specific tables in the tenant's database.

Start by copying the default User migration to the database/migrations/tenant directory:

cp database/migrations/0001_01_01_000000_create_users_table.php database/migrations/tenant

This will be the base migration for tenant-specific tables.

Implementing Tenant Routes

The tenancy package provides middleware to handle tenant-specific routes. This allows you to define routes that are accessible only to tenants and not to central domains.

Start by creating a new file routes/tenant.php for tenant-specific routes with the following content:

<?php

use Illuminate\Support\Facades\Route;
use Stancl\Tenancy\Middleware\InitializeTenancyByDomain;
use Stancl\Tenancy\Middleware\PreventAccessFromCentralDomains;

/*
|--------------------------------------------------------------------------
| Tenant Routes
|--------------------------------------------------------------------------
|
| Here you can register the tenant routes for your application.
| These routes are loaded by the TenantRouteServiceProvider.
|
| Feel free to customize them however you want. Good luck!
|
*/

Route::middleware([
    'web',
    InitializeTenancyByDomain::class,
    PreventAccessFromCentralDomains::class,
])->group(function () {
    Route::get('/', function () {
        return 'This is your multi-tenant application. The id of the current tenant is ' . tenant('id');
    });

    // Here you can add more tenant-specific routes
});

These routes will be loaded by the TenantRouteServiceProvider and will be accessible only to tenants. The InitializeTenancyByDomain middleware will set the current tenant based on the domain, and the PreventAccessFromCentralDomains middleware will prevent access from central domains.

For more information on how to customize the tenancy routes, refer to the stancl/tenancy documentation.

Implementing Tenant Creation

Create a controller for tenant registration, this would usually be done by the admin users of the application:

php artisan make:controller TenantController

Update the app/Http/Controllers/TenantController.php controller and implement the tenant registration process:

<?php

namespace App\Http\Controllers;

use App\Models\Tenant;
use Illuminate\Http\Request;

class TenantController extends Controller
{
    public function showRegistrationForm()
    {
        return view('tenant.register');
    }

    public function register(Request $request)
    {
        $request->validate([
            'domain' => 'required|string|max:255|unique:domains,domain',
        ]);

        $tenant = Tenant::create();
        $tenant->domains()->create(['domain' => $request->domain]);

        return redirect()->route('tenant.registered', $request->domain);
    }

    public function registered($domain)
    {
        return view('tenant.registered', compact('domain'));
    }
}

This controller handles tenant registration, creates a new tenant in the database, and sets up the tenant's domain. The TenancyServiceProvider will automatically map the tenancy events to the listener, which will create the tenant's database and run the tenant-specific migrations inside the database/migrations/tenant directory for the new tenant.

In a nutshell, the controller has three methods:

showRegistrationForm(): Displays the tenant registration form
register(): Registers a new tenant, which creates a new tenant record and domain
registered(): Displays a success message after registration

This controller will be used to manage tenant registration in our application. Allowing new tenants to register and create their own subdomain and database for their account.

Add routes for tenant registration in routes/web.php:

use App\Http\Controllers\TenantController;

Route::get('/register', [TenantController::class, 'showRegistrationForm'])->name('tenant.register');
Route::post('/register', [TenantController::class, 'register']);
Route::get('/registered/{domain}', [TenantController::class, 'registered'])->name('tenant.registered');

Create the corresponding views for tenant registration starting by creating the resources/views/tenant/register.blade.php file:

<x-guest-layout>
  <form method="POST" action="{{ route('tenant.register') }}">
    @csrf
    <div class="mt-4">
      <x-input-label for="domain" :value="__('Subdomain')" />
      <div class="flex">
        <x-text-input
          id="domain"
          class="mt-1 block w-full"
          type="text"
          name="domain"
          :value="old('domain')"
          required
        />
        <span class="text-gray-600 ml-2 mt-1">.example.com</span>
      </div>
    </div>

    <div class="mt-4 flex items-center justify-end">
      <x-primary-button class="ml-4"> {{ __('Register Tenant') }} </x-primary-button>
    </div>
  </form>
</x-guest-layout>

Then create the resources/views/tenant/registered.blade.php file to display the success message after registration:

<x-guest-layout>
  <div class="text-gray-600 mb-4 text-sm">
    {{ __('Your tenant has been registered successfully!') }}
  </div>

  <div class="mt-4 flex items-center justify-between">
    <div>
      Your tenant URL:
      <a
        href="https://{{ $domain }}.example.com"
        class="text-gray-600 hover:text-gray-900 text-sm underline"
        target="_blank"
        >https://{{ $domain }}.example.com</a
      >
    </div>
  </div>
</x-guest-layout>

This completes the tenant registration process. Tenants can now register and create their own subdomain and database for their account. In a real-world scenario, you would protect the registration routes with authentication middleware to ensure that only authorized admin users can create new tenants.

Verifying Tenant Registration

To verify that the registration process works, visit http://laravel-multi-tenant-saas.test/register and register a new tenant. After registration, you should see the success message with the tenant's domain.

Next go to your Neon dashboard and verify that the new tenant's database has been created:

SELECT * FROM tenants;

You should see the newly created tenant in the tenants table. You can also check the domains table to verify that the tenant's domain has been added:

SELECT * FROM domains;

And to verify that you actually have a separate database for the new tenant, use the \l command in the psql console to list all databases or the following SQL query:

SELECT datname FROM pg_database WHERE datistemplate = false;

The tenant's database should be listed in the results and it should be named tenant{tenant_id}.

The tenancy package allows you to configure the database naming convention for tenants. By default, the database name is tenant{tenant_id} where {tenant_id} is the ID of the tenant. You can also configure the package to use separate schemas instead of separate databases for tenants.

With that done, you've successfully implemented tenant registration in your multi-tenant SaaS application. Next let's implement the tenant onboarding process.

Implementing Tenant Onboarding

Now that you can register new tenants, let's create an onboarding process.

Each tenant will need to create an account to access their dashboard. The domain will be used to identify the tenant, so we'll use the domain as the tenant's subdomain, e.g., tenant1.example.com.

Create a new controller for tenant onboarding:

php artisan make:controller Tenant/OnboardingController

Update the app/Http/Controllers/Tenant/OnboardingController.php to handle the onboarding process:

<?php

namespace App\Http\Controllers\Tenant;

use App\Http\Controllers\Controller;
use App\Models\User;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Hash;

class OnboardingController extends Controller
{
    public function show()
    {
        if (User::count() > 0) {
            return redirect()->route('tenant.dashboard');
        }

        return view('tenant.onboarding');
    }

    public function store(Request $request)
    {
        $validated = $request->validate([
            'name' => 'required|string|max:255',
            'email' => 'required|string|email|max:255|unique:users',
            'password' => 'required|string|min:8|confirmed',
        ]);

        $user = User::create([
            'name' => $validated['name'],
            'email' => $validated['email'],
            'password' => Hash::make($validated['password']),
        ]);

        auth()->login($user);

        return redirect()->route('tenant.dashboard')->with('success', 'Welcome to your new account!');
    }
}

Add routes for the onboarding process in routes/tenant.php inside the Route::middleware group for tenant routes:

use App\Http\Controllers\Tenant\OnboardingController;

Route::middleware([
    'web',
    InitializeTenancyByDomain::class,
    PreventAccessFromCentralDomains::class,
])->group(function () {
    // Existing routes
    // ...

    Route::get('/onboarding', [OnboardingController::class, 'show'])->name('tenant.onboarding');
    Route::post('/onboarding', [OnboardingController::class, 'store'])->name('tenant.onboarding.store');

});

Create the onboarding view in resources/views/tenant/onboarding.blade.php:

<x-guest-layout>
  <form method="POST" action="{{ route('tenant.onboarding.store') }}">
    @csrf

    <div>
      <x-input-label for="name" :value="__('Name')" />
      <x-text-input
        id="name"
        class="mt-1 block w-full"
        type="text"
        name="name"
        :value="old('name')"
        required
        autofocus
        autocomplete="name"
      />
    </div>

    <div class="mt-4">
      <x-input-label for="email" :value="__('Email')" />
      <x-text-input
        id="email"
        class="mt-1 block w-full"
        type="email"
        name="email"
        :value="old('email')"
        required
        autocomplete="username"
      />
    </div>

    <div class="mt-4">
      <x-input-label for="password" :value="__('Password')" />
      <x-text-input
        id="password"
        class="mt-1 block w-full"
        type="password"
        name="password"
        required
        autocomplete="new-password"
      />
    </div>

    <div class="mt-4">
      <x-input-label for="password_confirmation" :value="__('Confirm Password')" />
      <x-text-input
        id="password_confirmation"
        class="mt-1 block w-full"
        type="password"
        name="password_confirmation"
        required
        autocomplete="new-password"
      />
    </div>

    <div class="mt-4 flex items-center justify-end">
      <x-primary-button class="ml-4"> {{ __('Complete Setup') }} </x-primary-button>
    </div>
  </form>
</x-guest-layout>

For simplicity, we're extending the Breeze guest layout for the onboarding form. But you can customize the layout to match your application's design and even have different layouts for the onboarding process based on each tenant's requirements.

To test the onboarding process, visit http://tenant1.example.com/onboarding and complete the onboarding form. After submitting the form, you should be redirected to the tenant dashboard which we'll implement next.

Implementing Tenant Dashboard

Create a new controller for the tenant dashboard:

php artisan make:controller Tenant/DashboardController

Update the app/Http/Controllers/Tenant/DashboardController.php to display the tenant dashboard:

<?php

namespace App\Http\Controllers\Tenant;

use App\Http\Controllers\Controller;
use Illuminate\Http\Request;

class DashboardController extends Controller
{
    public function index()
    {
        return view('tenant.dashboard');
    }
}

Create the dashboard view in resources/views/tenant/dashboard.blade.php:

<x-app-layout>
  <x-slot name="header">
    <h2 class="text-gray-800 text-xl font-semibold leading-tight">{{ __('Dashboard') }}</h2>
  </x-slot>

  <div class="py-12">
    <div class="mx-auto max-w-7xl lg:px-8 sm:px-6">
      <div class="overflow-hidden bg-white shadow-sm sm:rounded-lg">
        <div class="text-gray-900 p-6">{{ __("You're logged in!") }}</div>
      </div>
    </div>
  </div>
</x-app-layout>

Add a route for the tenant dashboard in routes/tenant.php inside the Route::middleware group for tenant routes:

use App\Http\Controllers\Tenant\DashboardController;

Route::middleware([
    'web',
    InitializeTenancyByDomain::class,
    PreventAccessFromCentralDomains::class,
])->group(function () {
    // Existing routes
    // ...
    Route::get('/dashboard', [DashboardController::class, 'index'])->name('tenant.dashboard');

});

To test the tenant dashboard, visit http://tenant1.example.com/dashboard after completing the onboarding process. You should see the dashboard view with a welcome message.

You can also check the users table in the tenant's database to verify that the user account created during onboarding has been added:

SELECT * FROM users;

This will show you the user account created during the onboarding process for that specific tenant in the tenant's database rather than the central database.

Conclusion

In this tutorial, we've built a simple multi-tenant application using Laravel and Neon. We've covered:

Setting up the project and implementing multi-tenancy
Creating a tenant registration process
Implementing tenant onboarding
Adding a tenant dashboard for individual tenants

This implementation provides a foundation for building more complex SaaS applications with Laravel and Neon. You can further expand on this system by:

Adding more features to the tenant dashboard
Implementing billing and subscription management
Enhancing security with two-factor authentication
Adding more tenant-specific customizations

Using the stancl/tenancy package along with Neon, each tenant will have its own database. Thanks to Neon's autoscaling feature, you can easily scale your application as you onboard more tenants.

There are other packages and tools available to help you build multi-tenant applications with Laravel. You can explore these options based on your requirements and choose the one that best fits your needs. Some of the popular packages include:

Additional Resources

How to use Synthetic Data to catch more bugs with Neosync

Rishi Raj Jain — Thu, 19 Dec 2024 15:06:37 +0000

A guide to developing robust, scalable and secure applications with Synthetic Data

In today’s world, developers are under more pressure than ever to develop robust, scalable and secure applications that have no bugs and are performant. Users want their applications and software to work flawlessly and any sign of a bug or disruption in workflow can lead to a frustrated or even lost user.

The challenge is that it’s really hard and time consuming to manually write test cases and test data in order to try to catch every edge case. Even then, it’s not perfect! A lot of teams will use their production data to test locally but this has a lot of privacy and security risks and by definition, you’re only using data that you’ve been able to successfully process. Which means that you’re not catching additional edge cases.

So then, as a developer, how can you be sure that you’re writing resilient code and developing a resilient application if you only have “happy path” data? The answer is that you really can’t. You need a better way to test and validate your systems.

This is where synthetic data can help you catch more bugs before you ship to production.

What is Synthetic Data?

Synthetic data is artificially generated data that is statistically and structurally just like real-world data, without containing any actual sensitive or personal information. Synthetic data is typically created using a combination of machine learning models and simulation algorithms.

This data can be nearly statistically identical to the source data set. For example, if the source data set has an age column with a range of 70 and a mean of 25.8, we can use machine learning models to create an identical column of brand new data that also has a range of 70 and mean of 25.8. Machine learning models such as CTGAN are great at learning the statistical properties of a set of data and replicating them with brand new data.

Further, with synthetic data, we can replicate the structure of the source data set. This means preserving the referential integrity or primary key and foreign key relationships, circular dependencies and other constraints.

Using a combination of machine learning and other algorithms we can replicate this structure in a data set such that you could easily import it into your current database without any problems. This is the power of synthetic data. The ability to create a data set that is statistically and structurally just like your production data without any of the privacy and security concerns.

Using Synthetic data to catch more bugs

Synthetic data can be a powerful tool to catch more bugs before they get reported in production. Let’s take a look at three of the most common use cases.

Generating diverse data to catch edge cases

Test driven development is more popular than ever which is a great thing for application resiliency. But the truth is that manually writing test cases can be tedious and really hard to anticipate every edge case. Most developers will test the happy path and then some edge cases before moving on.

This is where synthetic data can help. By generating a large and diverse set of data that would have taken days to manually write, you can check to make sure that your application can handle many different data types, formats and values.

The best part is that generating data can be really fast. For example, using Neosync, you can generate 10,000 rows of data for your Neon database in less than 30 seconds. It’s as simple as creating a connection to your Neon database and creating a job in Neosync. For more information, check out this blog post.

Ultimately, the end goal is that you can catch more bugs in your development and staging environments. Which means that you can ship a more resilient application.

Using Synthetic data for automated testing

Synthetic data can take automated testing to the next level. Developers can set up automated test suites in their CI pipelines and point to a database that is hydrated with synthetic data to test their applications. Instead of using the same stale data set, developers can have brand new data generated every time, increasing the chances that they catch a bug before they ship to production.

If they do catch a bug, then they can replicate that bug locally by hydrating a development or local database with the same data set and start to debug the error. This workflow of testing locally with synthetic data and then testing in CI with synthetic data, is a great way to catch bugs and gain confidence in your application’s resiliency.

Using Synthetic data to for performance testing

The last thing any developer wants is for their application to crash during a period of high-traffic. Imagine if Shopify crashed during Black Friday? For a company like Shopify, the problem is that they have a lot of sensitive data and can’t move their production data around easily without taking on a lot of privacy and security risk. So how do they get enough data for performance testing without worrying about privacy or security?

This is a perfect use case for synthetic data. Because synthetic data is artificially generated, you can create as much of it as you’d like. This makes performance testing a breeze and eliminates the privacy and security risks of handling production data. Just define how much data you need and then let the system create that data for you. You can then automatically ingest that into your application and monitor any performance issues without having to wait to see if it fails in production.

Conclusion

Synthetic data is a great tool that developers can use to catch more bugs and ship more resilient applications. It’s a powerful way to test your applications functionality and performance without any of the privacy and security concerns of production data. The great thing is that synthetic data tools and platforms are only getting better, higher fidelity and faster which makes it even more compelling for developers to start considering it as part of their core workflow.

Wrapping My Mind Around Node.js Runtimes

David Gomes — Fri, 22 Nov 2024 22:34:32 +0000

A look at Deno and Bun and how they work differently from Node.js

There’s been a lot of innovation in the last few years in Node.js runtimes. Node.js used to be the only viable option, but now there’s other contenders such as Bun and Deno. But are they ready for prime time? When should they be used, and by whom?

Wait, what even is a JavaScript runtime?

Good question. JavaScript wasn’t really built for the server. In fact, JavaScript wasn’t really “built” at all. What we think of as JS is just a specification.

A runtime is what brings that specification to life. It’s the environment that executes JavaScript code. When you’re working in a browser, the browser provides the runtime. But on a server, you need something else. That’s where Node.js came in and where these new runtimes are shaking things up.

A JavaScript runtime typically consists of several key components:

JavaScript Engine. This is the core that interprets and executes JavaScript code. V8 (used by Node.js and Chrome) is a famous example.
Event loop. Manages asynchronous operations, allowing non-blocking I/O operations.
APIs. Provides interfaces for system-level operations like file I/O, networking, and timers.
Standard library. A set of built-in modules and functions that developers can use out of the box.

Node’s ability to handle thousands of concurrent connections with its non-blocking I/O model has made it particularly popular for real-time applications, APIs, and microservices architectures. And it is still the most popular “web technology” according to the 2024 Stack Overflow survey:

The problems with Node.js

But Node is not without its detractors—including the person who initially built Node, Ryan Dahl. In a 2019 jsconf talk titled 10 Things I Regret About Node.js, Dahl lists the problems he now sees in Node, such as…

Security. By default, the module system in Node.js allows any package to access the file system, network, and environment variables. This poses significant security risks, especially when using third-party modules.
Package management. The package.json file, npm, and the Node modules system have become overly complex. There’s also various security issues around npm (like the postinstall scripts).
Build system complexity. Node’s build system, particularly for native modules, has become unnecessarily complicated over time. Basically, supporting both CommonJS and ES modules made Node.js much more complex.

So, all this together has led a few people to think they can do better.

The new runtimes in town

Deno: Security-oriented, modern JavaScript

Ryan Dahl chose to take the learnings from Node and built Deno. Deno addresses many of the concerns he had with Node.js:

Focus on security. Deno runs code in a sandbox by default. Accessing the file system, network, or environment variables requires explicit permissions. This approach significantly reduces the risk of malicious code execution.
Built-in TypeScript support. Unlike Node.js, which requires additional setup for TypeScript, Deno supports TypeScript out of the box. This integration streamlines development and improves code quality.
Standard modules. Deno provides a standard library of high-quality, audited modules. This reduces dependency on third-party packages for common functionalities.
Browser compatibility. Deno aims to be as close as possible to browser JavaScript. It uses web standard APIs where possible, making it easier for developers to write code that works both in the browser and on the server.
Import URLs. Instead of using a package manager like npm, Deno allows modules to be imported directly via URLs.
This will somewhat change with Deno v2, more on this later.
Single executable. Deno ships as a single executable with no external dependencies, making it easy to install and use across different environments.

Deno is gaining traction, especially among developers who appreciate its modern features and security-first approach. It’s particularly attractive for projects that can benefit from its TypeScript integration and don’t rely heavily on Node-specific libraries.

Deno also provides infrastructure beyond the runtime. Tests, linters, and formatters are also included. You can also deploy your applications using Deno’s serverless environment, Deno Deploy (learn how to use Neon with Deno Deploy), and build entire applications using Deno’s framework, Fresh.

Here’s a simple setup with Deno to use Neon. The Deno-specific things are the import (@neon/serverless is called @neondatabase/serverless in NPM) and the Deno.env.get bits. Also, this is a Deno v2 example (which is currently in “canary”).

import { neon } from "@neon/serverless";

// Get the connection string from the environment variable "DATABASE_URL"
const databaseUrl = Deno.env.get("DATABASE_URL")!;

if (!databaseUrl) {
  throw new Error(`Please add a DATABASE_URL environment variable.`);
}

const sql = neon(databaseUrl);

const rows = await sql`SELECT version()`;

console.log(rows[0].version);

To run this, call:

DATABASE_URL=$DATABASE_URL deno test.ts

(And notice how Deno will ask for permission to read environment variables and make network requests at runtime.)

To show the speed of progress with Deno, up until a few months ago, the above code would have used an import URL, such as:

import * as postgres from 'https://deno.land/x/postgres@v0.17.0/mod.ts';

Deno just released version 1.46, which will be the last 1.X version. Deno 2 is changing in response to feedback from the community and a greater understanding of the initial strict choices of the runtime.

Take imports. Originally the idea was to have simple URLs like https://deno.land/x/postgres@v0.17.0/mod.ts. As Deno put it:

Designing Deno’s module system around HTTP imports was ambitious. It aimed to replace npm with a distributed system over HTTP, aligning with how ES Modules work in browsers. This eliminated the need for package.json files and node_modules folders, simplifying project structures.

But the reality wasn’t as clean. Deno found that dependency management, clutter, and reliability were all a problem with this approach. So, they’ve launched JSR, the JavaScript Registry. This is akin to npm in node, but TypeScript and ECMAScript-native. All you have to do is:

deno add @neon/serverless

Then you’re ready to roll.

Like Node.js, Deno is built on the V8 engine. Originally, Deno wasn’t supposed to be compatible with Node.js. But, as with import URLs, pragmatism beat out idealism. The Deno team realized that compatibility with the vast Node.js ecosystem would significantly increase Deno’s adoption and usefulness. As a result, they’ve been working on Node.js compatibility layers, allowing developers to use many Node.js packages and APIs within Deno projects, thereby bridging the gap between the two runtimes.

If you want to learn more, I highly recommend the recent participation of Ryan Dahl on the “Syntax” podcast to learn more about Deno v2.

Bun: A performance-focused Node replacement

Bun is a little different. It was Node-compatible out of the gate and not looking to right-the-wrongs of Node.js. But, as it grew, it started to evolve towards a slightly different direction.

Benchmarking is hard. We at Neon are definitely very well aware of that, but my anecdotal experience is that Bun is indeed really fast. How fast exactly is very hard to measure, of course.

Like Deno, Bun is looking to rely on JavaScript primitives and code simplicity. Here’s Neon running with Bun:

import { neon } from "@neondatabase/serverless";

const databaseUrl = process.env.DATABASE_URL;

if (!databaseUrl) {
  throw new Error(`Please add a DATABASE_URL environment variable.`);
}

const sql = neon(databaseUrl);

const rows = await sql`SELECT version()`;

console.log(rows[0].version);

To call this, run:

DATABASE_URL=$DATABASE_URL bun test.ts

Bun’s approach to improving the JavaScript runtime environment focuses on several key areas:

Performance. Built on the JavaScriptCore engine (used by Safari), Bun is designed for speed from the ground up. Its fast startup times and efficient execution make it particularly suitable for serverless environments and microservices.
TypeScript support. Similar to Deno, Bun offers out-of-the-box TypeScript support. However, Bun takes it a step further by transpiling TypeScript at runtime, eliminating the need for separate build steps in many cases.
Enhanced APIs. Bun provides additional APIs that aren’t available in Node.js, such as the Bun.file API for efficient file operations and the SQLite API for built-in database functionality.
Streamlined configuration. Bun aims to reduce configuration overhead. For instance, it automatically loads environment variables from .env files, simplifying local development and deployment processes.
All-in-one toolkit. Bun isn’t just a runtime; it’s a complete toolkit. It includes a package manager, test runner (which Node.js now has as well), and bundler, all optimized for performance. This integrated approach simplifies the development workflow and reduces the need for additional tools.

The Bun team is rapidly iterating and adding some interesting features. For example, they introduced “hot module reloading” for server-side code, a feature that’s been long desired in the Node.js ecosystem. Also, more recently, they added support for C code using TinyCC 🤯And I believe they led the way with SQLite embedding (but Node.js has now caught up).

While Bun is still young compared to Node.js and even Deno, its focus on performance and developer experience is attracting attention. It is particularly appealing for projects where raw speed is a priority, such as high-traffic web servers or compute-intensive apps.

Furthermore, it’s also extremely appealing to use Bun not as your production runtime, but just as your package manager. We’re currently doing that with neonctl, purely because of Bun’s performance (we also want to migrate the CLI runtime to Bun at some point, due to its single-file executable feature and overall performance).

Final thoughts

First of all, I have to mention that there are a few other runtimes that are probably worth mentioning. Although, perhaps it’d be better to refer to some of them as “neo-runtimes” since they’re not necessarily full blown runtimes:

Then, there’s the aspect of picking a runtime. This is a very hard decision and obviously migrating existing projects is even harder. But this is my very personal current thought process:

Use Node.js for anything that needs to live for really long, or doesn’t need Deno’s security model
Use Deno if you want extra security, or if you specifically want to use Deno Deploy.
Don’t use Deno if all you care about is not transpiling TypeScript. Node.js will support TypeScript soon enough.
Use Bun if performance is the most important thing to you. Or, if you want to use some of Bun’s more “fun” features (Macros, embedded SQLite, etc.) and you don’t have a super critical requirement that your project outlives your time on Earth (Bun is afterall backed by a very early stage company).

That’s basically it, but I know this is all very subjective — do your own research! I really love Bun’s performance and I try to use Bun for all package management on my local machine. However, I would only use Bun as a production runtime under certain circumstances for now.

If you’re looking for a more pessimistic take on new runtimes, I can recommend this blog post. But I tend to have a more positive outlook on new companies building cool new dev tooling.

Looking at How Replit Agent Handles Databases

Sam シ — Wed, 13 Nov 2024 16:16:59 +0000

Replit Agent is a powerful tool within the Replit development environment that allows you to offload coding tasks using natural language and AI. By interacting with the Replit Agent, you can generate code, create applications, and modify existing features simply by asking for what you need.

This automation becomes particularly valuable when you’re working on full-stack applications that require database changes, where it helps maintain consistency between the code and the database schema. Essentially, you can have the Replit Agent implement end-to-end features within your application, changing the API, Database, and Frontend all at once. Getting Replit Agents to handle database migrations

Introducing the tools

To manage your app’s data, Replit provisions a database, a fully managed serverless Postgres database (powered by Neon) that scales on demand. The Agent builds the rest of the app in Flask, a python microframework for web applications which supports API endpoints and HTML templating.

With a little prompting, you can get the Agent to use Alembic, a database migration tool that works with SQLAlchemy. SQLAlchemy acts as an Object Relational Mapper (ORM), which helps convert your Python objects and code into the schema used by your Postgres database. Alembic tracks changes to your SQLAlchemy models and automatically generates migration scripts for database schema updates.

In this article, we will explore how this process works by creating a simple TODO application where features are added incrementally using only natural language prompts to the Replit Agent. You’ll see how you can get Alembic migrations to be automatically generated and how changes are reflected in your Neon database.

To follow along, you need access to the Replit Agent, which is available on the Replit Core or Team subscriptions.

Creating the project

To begin, head over to your Replit home dashboard and create a new Repl. Instead of setting everything up manually, simply describe the project you want in the Replit Agent textbox. To follow this article, ask the agent to:

create me a TODO application API with CRUD functionalities and a simple frontend to view the tasks

Once you describe the project, Replit Agent will generate a plan that includes the core features and may also suggest additional features or enhancements that could be useful. You can review these suggestions and make adjustments if needed. After you approve the plan, the agent will provision a Neon PostgreSQL database, generate the SQLAlchemy schema for your application, set up the Flask API endpoints, and create the HTML and JavaScript for the frontend.

Within a minute or two, the Replit Agent will finish setting up the fully functional application. At this point, it will prompt you to test everything to make sure it’s working as expected. You should try adding a simple task to the app and confirm that the basic functionality is correct. Once you verify that everything is in order, the Agent will create a commit and a checkpoint for the application.

Adding a due date feature

With the basic TODO app up and running, you can extend its functionality by allowing users to optionally specify a due date for each task to help with setting deadlines or reminders. To add this feature, simply ask the Replit Agent:

can you add another text box so that the user can optionally add a due date to the task

The Replit Agent understands that this feature requires changes to both the frontend (adding a date input) and the backend (modifying the database schema to store the due date). Specifically, a new column needs to be added to the tasks table to store the due_date.

Here’s how the Agent handles the database update process:

Generating the SQLAlchemy model update

The agent modifies the Task model to include a new due_date field. The updated model now looks like this:

    class Task(db.Model):
        id = db.Column(db.Integer, primary_key=True)
        title = db.Column(db.String(100), nullable=False)
        description = db.Column(db.Text, nullable=True)
        completed = db.Column(db.Boolean, default=False)
        due_date = db.Column(db.DateTime, nullable=True)

Creating and applying alembic migrations

Since this change alters the database schema, the Replit Agent generates a migration script using Alembic. The migration will add the due_date column to the tasks table in the database. Here’s the up migration script that adds the due_date column:

  """Add due_date to Task model

    Revision ID: 0f4af0e04d02
    Revises:
    Create Date: 2024-10-15 17:40:22.891574

    """
    from alembic import op
    import sqlalchemy as sa


    # revision identifiers, used by Alembic.
    revision = '0f4af0e04d02'
    down_revision = None
    branch_labels = None
    depends_on = None


    def upgrade():
        # ### commands auto generated by Alembic - please adjust! ###
        with op.batch_alter_table('task', schema=None) as batch_op:
            batch_op.add_column(sa.Column('due_date', sa.DateTime(), nullable=True))

        # ### end Alembic commands ###


    def downgrade():
        # ### commands auto generated by Alembic - please adjust! ###
        with op.batch_alter_table('task', schema=None) as batch_op:
            batch_op.drop_column('due_date')

        # ### end Alembic commands ###

Notice that it provides both an upgrade and downgrade migration such that it can also handle rollbacks to previous states.

Frontend update

The Agent also updates the frontend to include a new text box where users can input a due date when creating or updating a task, as seen here:

After you confirm the update, the due_date field is now part of both the UI and the database, and the agent ensures the changes are committed properly.

Adding more features: Priority field

You can continue adding features to your TODO app in a similar way. For example, to help users manage the importance of their tasks, you might want to add a priority field to each task. Like before, asking Replit Agent to implement this will trigger another schema change, and it will generate and apply the relevant Alembic migrations automatically:

add a priority flag to the task creation that can be low, medium, or high and that shows up on the tasks list
Once the agent has processed the request, it will update both the backend and frontend. The priority field will now appear in the task creation form, and tasks will display their priority (low, medium, or high) in the tasks list:

At this point, you have introduced two schema changes to the database: one for the due_date field and one for the priority field. With each change, Alembic generated a separate migration file. These migration files represent individual checkpoints in the evolution of your database schema.

When you look into the migrations/versions folder of your project, you’ll now see two distinct migration files:

{commit-id}_add_due_date_to_task.py – This migration was created when the due_date field was added.
{commit-id}_add_priority_to_task.py – This migration was created when thepriorityfield was added.

These migration files are important because they allow you to easily rollback your database schema to any previous state. For example, if you want to undo the addition of the priority field but keep the due_date, you can apply a rollback to the previous migration using Alembic’s rollback functionality.

By having separate migrations for each change, you can always track the evolution of your schema and revert specific changes when needed.

Rolling back

There may come a time when you’re not happy with how a feature turned out, or you simply don’t need it anymore. This is where Replit’s rollback feature, powered by Alembic, becomes especially useful. Not only can you revert the application to a previous state, but you can also undo the database changes, ensuring everything stays consistent.

In this case, let’s say you no longer need the priority field or you want to remove it for some reason. Rolling back the application and database to the state before the priority field was added is straightforward.

Replit Agent will automatically run the downgrade migration script, and the priority field will be removed from the database. Additionally, the app’s frontend will be updated, and the feature will no longer appear in the task creation form or task list.

Here’s the UI after the rollback, which now looks just like it did before the priority field was added:

To confirm that the database has been successfully rolled back, you can use the Postgres tool in the Replit development environment to inspect the tasks table. As shown below, the priority column is no longer present:

This means the database schema has been fully reverted to its earlier state, and the feature has been removed not just from the code, but also from the database. Because Alembic tracks every migration in the project, you can roll back your schema to any previous migration—whether that’s the most recent change or a much earlier one. The migration files in the migrations folder act like checkpoints in your database’s timeline. Each migration has a unique revision ID, which allows you to target a specific state when rolling back.

Conclusion

Replit Agents can use Alembic and Neon PostgreSQL to handle database changes efficiently. When you request a new feature that affects the database, you can get the agent to generate a migration files with two functions—one for applying changes and another for rolling them back. This makes it easy to update or revert your database schema as needed.

By creating these files with each change, Replit ensures that your database stays in sync with your application. Whether you’re adding new fields, removing features, or undoing changes, the agent manages the migrations so you don’t have to worry about keeping the database consistent. This allows you to focus on building your application while the underlying structure is handled for you.

Neon is the serverless Postgres database backing Replit Agents. Start using it for free.

DEV Community: Neon

Interact with Neon APIs using Claude Code through natural language

Key components

Setting up Neon MCP Server in Claude Code

Prerequisites

Option 1: Setting up the Remote Hosted Neon MCP Server

Option 2: Setting up the Local Neon MCP Server

Verification

Supported Actions (Tools)

Development Workflow: Adding a Column with Claude Code and Neon MCP

Bonus: Creating a project and branch

Creating a new project

Creating a New Branch

Conclusion

Resources

Build Internal Tools Better: Using Neon, StackAuth, and Vercel

Getting Started with the Template

Set Up Roles and Claim Your Project

Authentication and Access Control with StackAuth

Database and Logic with Neon and Next.js

Customize the App for Your Team

Get Started

Building Intelligent Search with AI Embeddings, Neon, and pgvector

What you'll build

Prerequisites

Understanding AI embeddings and vector search

What are embeddings?

Why pgvector?

Setting up pgvector on Neon

Building an embedding generation service

Creating a document management system

Building the search API

Testing the semantic search system

Performance optimization

Conclusion

Additional Resources

app.build – Open-Source AI Agent That Builds Full-Stack Apps

How can I try it then?

Can I run this locally?

What features does the platform offer?

How does the agent work?

How can I get involved?

Comparing Text Search Strategies pg_search vs. tsvector vs. External Engines

Built-in PostgreSQL tsvector for Text Search

Example of Using tsvector

Pros of tsvector

Cons of tsvector

Extending tsvector with pg_search

Enabling pg_search on Neon

Example of Using pg_search

Why Use pg_search on Neon?

External Search Engines (e.g., Elasticsearch)

Use Cases for External Search Engines

Comparison Summary

Which Option Should You Choose?

Use built-in tsvector when:

Use pg_search on Neon when:

Consider external search engines when:

Conclusion

Coding With Cursor and Windsurf Side by Side

What feels the same?

Tab Completions: Cursor’s Tab vs. Windsurf’s Tabs

Context management: Attaching the relevant information

Agent mode: Cursor’s Composer vs Windsurf’s Cascade

Conclusion

Building a Multi-Step Form with Laravel Volt, Folio, and Neon Postgres

Prerequisites

Setting up the Project

Configuring the Database Connection

Database Design

Creating Models

Creating a layout for the multi-step form

Implementing File-based Routing with Folio

Building the Multi-Step Form with Volt

Personal Information Form

Education Form

Work Experience Form

Review Form

Confirmation Page

Testing the Multi-Step Form

Built-in PostgreSQL `tsvector` for Text Search

Example of Using `tsvector`

Pros of `tsvector`

Cons of `tsvector`

Extending `tsvector` with `pg_search`

Enabling `pg_search` on Neon

Example of Using `pg_search`

Why Use `pg_search` on Neon?

Use built-in `tsvector` when:

Use `pg_search` on Neon when:

Enabling Swagger in `Program.cs`