DEV Community: Konstantinos Passadis | Azure MVP | MCT

The Rise of vLLM in Modern Cloud Development: Revolutionizing AI Inference

Konstantinos Passadis | Azure MVP | MCT — Sun, 01 Mar 2026 16:38:24 +0000

The Rise of vLLM in Modern Cloud Development: Revolutionizing AI Inference

Introduction to the AI Cloud Bottleneck The emergence of Large Language Models (LLMs) has revolutionized cloud applications, from universal chatbots to automated programming assistants. However, hosting these models in the cloud is notoriously expensive due to the massive computational and memory re

linkedin.com

Make your Applications smarter: Powerful Machine Learning demo

Konstantinos Passadis | Azure MVP | MCT — Thu, 06 Nov 2025 23:32:36 +0000

cloudblogger.eu

Orbit v0.4.5

Konstantinos Passadis | Azure MVP | MCT — Thu, 06 Nov 2025 23:24:02 +0000

🚀 Orbit v0.4.5 - Next Gen Distributed VCS with Email-Based Namespaces

Orbit is a production-ready distributed version control system built on the revolutionary Virtual Object Store (VOS) architecture with VNP (VOS Network Protocol) for lightning-fast, SHA3-secured transactions. Features AI-powered natural language queries, email-based namespace security, self-service registration, and GitHub-compatible clone workflows with seamless distributed development and auto-repository creation.

🎯 Key Features (v0.4.5)

🤖 AI-Powered Queries - Ask questions about your repository in natural language using Azure OpenAI
⚡ Revolutionary VOS Architecture - Virtual Object Store with 40% faster operations than Git
🌐 VNP Protocol - Custom VOS Network Protocol with SHA3-256 secured transactions
📧 Email-Based Namespaces - alice@company.com gets alice/* access (collision-proof)
🔐 Self-Service Registration - REST API user management with token authentication
🏗️ Auto-Repository Creation - Repositories created automatically when accessed
📥 GitHub-Like Clone Workflow - orb clone → orb checkout → actual files
🔄 Complete Object Graph Sync - Full commits, trees, files, and chunks synchronization
☁️ Azure Production Deployment - Container Apps with persistent namespace storage

You want to test it end to end ? Contact me via passadis.github.io to discuss your use case and help you set up the Server!

🚀 Quick Start

# 1. Register with email (self-service)
curl -X POST http://your-server.com:8081/admin/users \
  -H "Content-Type: application/json" \
  -d '{"username": "alice@company.com", "repositories": [], "permissions": {"read": true, "write": true, "admin": false}}'

# 2. Set authentication token
export ORBIT_TOKEN="your-token-here"

# 3. Clone any repository (auto-created if doesn't exist)
orb clone "orbits://your-server.com:8082/alice/my-project" my-project

# 4. Checkout files and start working
cd my-project
orb checkout

# 5. Ask AI about your repository! 🤖
orb ai "What files are tracked in this repository?"
orb ai "What is the current branch and latest commit?"
orb ai "How many files were changed recently?"

🤖 AI-Powered Queries (NEW!)

Orbit now features built-in AI assistance powered by Azure OpenAI!

Ask questions about your repository in natural language:

# Query repository state
orb ai "What is the current branch?"
orb ai "How many files are tracked?"
orb ai "What's the latest commit message?"

# Analyze changes
orb ai "What files were modified recently?"
orb ai "Tell me about recent commits"
orb ai "Is the repository clean?"

Setup: Create .orbit-ai-config.json or use environment variables:

AZURE_OPENAI_ENDPOINT - Your Azure OpenAI endpoint
AZURE_OPENAI_API_KEY - Your API key
AZURE_OPENAI_DEPLOYMENT - Model deployment name
AZURE_OPENAI_API_VERSION - API version

The AI has full context of your repository including current branch, tracked files, commit history, and working directory status!

📧 Email-Based Security

Access repositories based on your email namespace:

alice@company.com → can access alice/* repositories
bob@startup.io → can access bob/* repositories
Automatic collision prevention and namespace isolation

🏗️ Self-Service Repository Management

New in v0.4.5: Repositories are created automatically when you access them:

# List your namespace repositories (authenticates automatically)
orb list-repos "orbits://your-server.com:8082"

# Clone creates repository if it doesn't exist
orb clone "orbits://your-server.com:8082/alice/new-idea" new-idea

# Your email determines namespace access:
# alice@company.com can access alice/project1, alice/project2, etc.

☁️ Azure Production Ready

Deploy with complete namespace isolation and persistent storage:

# Server runs on port 8082 (VNP protocol)
# Admin API runs on port 8081 (user management)
# Each namespace gets isolated directory: /alice/, /bob/, etc.

🔧 Command Reference

Core Commands

orb init                           # Initialize new repository
orb save -m "message"              # Create commit with complete object graph
orb check                          # Check working directory status
orb history                        # Show commit history (DAG)
orb revert                         # Revert files to their last committed state
orb fetch                          # Fetch and convert a Git repository to Orbit format
orb checkout                       # Checkout files from commits

AI Commands (NEW in v0.4.5) 🤖

orb ai "your question"             # Ask AI about your repository using natural language
                                   # Examples: "What's the current branch?"
                                   #          "How many files are tracked?"
                                   #          "What's the latest commit?"

Distributed Commands (v0.4.5)

orb list-repos <url>               # List repositories in your namespace
orb clone <url/namespace/repo> <local-name>  # Clone (auto-creates if needed)
orb sync <url>                     # Synchronize with remote server
orb register                       # Register a new user account on an Orbit server

🏗️ Architecture

Revolutionary VOS + VNP Architecture

Virtual Object Store (VOS) - 40% faster than Git with content-addressed storage
VNP Protocol - Custom VOS Network Protocol with SHA3-256 secured transactions
Post-Quantum Security - SHA3-256 hashing for future-proof cryptographic security
Email-Based Namespaces - alice@company.com → alice/* access with collision prevention
Auto-Repository Creation - Repositories created on first access with namespace isolation
AI Integration - Azure OpenAI GPT-4o for natural language repository queries

Production Deployment

Azure Container Apps - Dual-port deployment (8082 + 8081)
Namespace Isolation - Each user gets isolated directory
REST Admin API - Self-service user registration
TLS Security - End-to-end encrypted communication
AI-Powered CLI - Built-in intelligent assistant for repository operations

📋 Version History

🚀 v0.4.5 - Email-Based Namespaces & AI-Powered Queries (Current)

Released: November 2025

🤖 AI-Powered Queries - Natural language repository queries using Azure OpenAI GPT-4o
📧 Email-Based Namespace Security - alice@company.com gets alice/* access automatically
🔐 Self-Service Registration - REST API for user management without admin intervention
🏗️ Auto-Repository Creation - Repositories created when accessed (like GitHub)
📥 Complete Clone Workflow - orb clone → orb checkout → working files extracted
🔄 Object Graph Integrity - Full commits, trees, files, chunks with proper routing
☁️ Production Azure Deployment - Dual-port server with namespace isolation

Orbit v0.4.5 - AI-powered distributed VCS with email-based security and natural language queries. 🌟🤖

Built with ❤️ by K.Passadis in Rust for performance, security, and developer productivity.

Orbit v0.3.0 - Next-Generation Version Control System

Konstantinos Passadis | Azure MVP | MCT — Tue, 14 Oct 2025 18:49:03 +0000

🚀 Orbit - Next-Generation Version Control System

Orbit is a performance-focused, post-quantum secure version control system built on a revolutionary Virtual Object Store (VOS) architecture. Designed for the future of software development, Orbit delivers superior performance while maintaining cryptographic security against quantum computing threats.

⚡ Key Performance Advantages

40% Faster Status Checks - VOS Index optimization with metadata caching
1.75x-2.67x Overall Performance - Benchmarked against Git with statistical validation
Lightning-Fast Operations - Selective re-hashing and intelligent caching
Post-Quantum Security - SHA3-256 (Keccak) cryptographic hashing throughout

🛡️ Security & Architecture

Post-Quantum Cryptography

SHA3-256 (Keccak) hashing for all objects and commits
Future-proof against quantum computing attacks
NIST-approved cryptographic standards

Virtual Object Store (VOS)

Content-Defined Chunking using FastCDC algorithm
Global Deduplication across repository history
Efficient Storage with intelligent object compression
Metadata-Based Optimization for instant status checks

🎯 Revolutionary VOS Index

Orbit's VOS Index represents a breakthrough in version control efficiency:

Metadata Caching - File attributes cached for instant comparison
Selective Re-hashing - Only modified files are processed
Timestamp Intelligence - Smart file change detection
Zero-Copy Operations - Minimal I/O for status checks

This novel approach significantly outperforms traditional index mechanisms used by Git and Mercurial.

📦 Installation

# Install from source (Rust required)
git clone https://github.com/your-org/orbit
cd orbit
cargo install --path .

# Verify installation
orb --version

🚀 Quick Start

# Initialize a new repository
orb init

# Check repository status (40% faster than git status)
orb status

# Save changes with a commit
orb save -m "Initial commit with post-quantum security"

# View commit history with DAG visualization
orb history

# Revert files to last committed state
orb revert README.md

� Migrating from Git

Orbit v0.3.0 makes Git migration seamless! Convert any Git repository to Orbit format with full history preservation:

# Migrate any Git repository (local or remote)
orb fetch https://github.com/user/repository.git

# Specify custom target directory
orb fetch --target my-project https://github.com/user/repository.git

# Navigate and use Orbit commands
cd repository
orb status    # 40% faster than git status
orb history   # View converted commit history

What gets preserved:

✅ Full commit history with SHA3-256 security upgrade
✅ Author information and timestamps
✅ Commit messages and metadata
✅ File contents with content-defined chunking
✅ Directory structure exactly as in Git

What gets upgraded:

🔐 Post-quantum security with SHA3-256 hashing
⚡ Performance improvements with VOS Index optimization
📦 Better deduplication with FastCDC chunking

�📊 Benchmarked Performance

Operation	Git	Orbit v0.2	Improvement
Status Check	110.3ms ± 20.6ms	63.1ms ± 25.6ms	1.75x faster
Initial Commit	1.694s ± 0.028s	1.057s ± 0.199s	1.60x faster
Repository Init	~50ms	~30ms	1.67x faster

Benchmarks performed with hyperfine statistical analysis on realistic codebases

🔧 Command Reference

Core Commands

orb init                    # Initialize new repository
orb save -m "message"       # Create commit with message
orb status                  # Check working directory status
orb history                 # Show commit history (DAG)
orb revert [files...]       # Revert files to HEAD state

Information Commands

orb --help                  # Comprehensive help system
orb --version               # Show version information
orb <command> --help        # Command-specific help

Advanced Features (Coming Soon)

orb sync                    # Remote synchronization (v0.3+)
orb branch                  # Branch management (v0.3+)
orb merge                   # Intelligent merging (v0.3+)
# Additional advanced features in development...

🏗️ Technical Architecture

Object Model

Commits - DAG nodes with SHA3-256 integrity
Trees - Directory structures with chunked content
Blobs - File data with content-defined chunking
Index - Metadata cache for performance optimization

Storage Engine

Content Addressing - All objects identified by SHA3-256 hash
Deduplication - Identical content stored only once globally
Compression - Efficient storage with modern algorithms
Integrity - Cryptographic verification of all data

Performance Optimizations

VOS Index Caching - Metadata-based change detection
Selective Processing - Only modified files are re-processed
Parallel Operations - Multi-threaded where beneficial
Zero-Copy I/O - Minimal data movement for speed

🔬 Innovation Highlights

Novel VOS Index Implementation

Orbit's VOS Index uses advanced metadata caching combined with selective re-hashing to achieve 40% faster status checks compared to traditional version control systems. This innovative approach caches file metadata and performs intelligent timestamp-based change detection, eliminating unnecessary hash computations.

Integrated Content-Defined Chunking

The seamless integration of FastCDC (Content-Defined Chunking) with SHA3-256 post-quantum cryptography within the VOS object model enables:

Global deduplication across entire repository history
Efficient storage of large binary files
Future-proof security with quantum-resistant hashing
Optimal performance with intelligent chunking boundaries

🛣️ Roadmap

v0.3 - Distributed Operations

Remote repository synchronization
Branch management and merging
Advanced conflict resolution
Network protocols for collaboration

v0.4 - Enterprise Features

Access control and permissions
Repository analytics and insights
Advanced merge strategies
Performance monitoring

v1.0 - Production Ready

Full Git compatibility layer
Migration tools and utilities
Enterprise deployment tools
Comprehensive documentation

📈 Why Choose Orbit?

For Developers

Faster Operations - Spend less time waiting, more time coding
Modern Architecture - Built with current best practices
Future-Proof - Post-quantum cryptography ready
Intuitive Commands - Clean, discoverable interface

For Organizations

Performance Gains - Measurable productivity improvements
Security Assurance - Quantum-resistant cryptography
Innovation - Next-generation version control technology
Reliability - Rust-based implementation with memory safety

🤝 Contributing

Orbit is under active development. We welcome contributions in:

Performance optimizations
Security enhancements
Feature development
Documentation improvements
Testing and validation

📄 License

MIT License - See LICENSE file for details.

🔗 Links

Documentation: [Coming Soon]
Issues: GitHub Issues
Discussions: GitHub Discussions
Benchmarks: Included in repository under /benchmarks

📋 Version History

🚀 v0.3.0 - Git Interoperability (Current)

Released: October 2025

🔄 Git Migration: New orb fetch command for seamless Git-to-Orbit conversion
🌐 Repository Import: Import any Git repository with full history preservation
🧹 Smart Cleanup: Windows-compatible file handling and cleanup
⚡ In-Place Conversion: Efficient conversion process without temporary directories
📊 Migration Stats: Real-time progress indicators during conversion
🔒 Preserved Metadata: Author information, timestamps, and commit messages maintained

🏗️ v0.2.0 - Foundation Release

Released: October 2025

🔐 Post-Quantum Security: SHA3-256 (Keccak) cryptographic hashing
⚡ VOS Index: 40% faster status checks with metadata optimization
📦 FastCDC Chunking: Content-defined chunking for deduplication
🎯 Core Commands: init, save, status, history, revert
📈 Performance Benchmarks: Comprehensive performance testing suite
🛡️ Data Integrity: Tamper-proof commit signatures and verification

🌱 v0.1.0 - Initial Concept

Released: Early 2025

💡 Proof of Concept: Basic version control functionality
🏛️ DAG Architecture: Directed Acyclic Graph for commit relationships
📁 Virtual Object Store: Foundation VOS implementation
🦀 Rust Implementation: Memory-safe systems programming foundation

🎯 What's Next?

🔮 v0.4.0 - Remote Collaboration (Planned)

☁️ Remote Repositories: Push/pull with remote Orbit repositories
🤝 Conflict Resolution: Advanced merge algorithms
🔄 Sync Protocol: Efficient synchronization between repositories

🔮 v0.5.0 - Advanced Features (Planned)

🌿 Branch Management: Full branching and merging support
🏷️ Tagging System: Release tagging and versioning
📊 Advanced Analytics: Repository statistics and insights

Orbit v0.3.0 - Git migration made simple. The future of version control is here. 🌟

Built with ❤️ by @passadis in Rust for performance, security, and developer productivity.

How to built your Custom AI Server with AKS and your custom Clients with Vite and NextJS – Part 1

Konstantinos Passadis | Azure MVP | MCT — Mon, 31 Mar 2025 22:39:30 +0000

cloudblogger.eu

Azure AI Agent Service The brand new Azure AI Agent Service at your fingertips

Konstantinos Passadis | Azure MVP | MCT — Mon, 31 Mar 2025 22:36:10 +0000

cloudblogger.eu

How to orchestrate your Agents with Azure AI Agent Service and Azure API Management

Konstantinos Passadis | Azure MVP | MCT — Thu, 27 Feb 2025 00:04:37 +0000

The brand new Azure AI Agent Service at your fingertips - CloudBlogger@2025

Azure AI Agent Service is the newest addition in Azure AI Foundry, making the process of creating Agents easier and fun ! Lets see that in action.

cloudblogger.eu

Azure Index AI & MS Fabric

Konstantinos Passadis | Azure MVP | MCT — Wed, 18 Sep 2024 19:26:02 +0000

How to create an AI Web App with Azure OpenAI, Azure AI Search with Vector Embeddings and Microsoft Fabric Pipelines

Intro

Welcome, visitor! Today, we embark on an exciting journey to build an AI Assistant and Recommendations bot with cutting-edge features, helping users decide which Book is best suitable for their preferences. Our bot will handle various interactions, such as, providing customized recommendations, and engaging in chat conversations. Additionally, users can register and log in to this Azure Cloud-native AI application. Microsoft Fabric will handle, automation and AI related tasks such as:

Load and clean the books Dataset with triggered Pipelines and Notebooks
Transform the Dataset to JSON and making proper adjustments for Vector usability
Load the cleaned and transformed Dataset to Azure AI Search and configuring Vector and Semantic profiles
Create and save embeddings with Azure OpenAI to Azure AI Search

As you may already guessed our foundation lies in Microsoft Fabric, leveraging its powerful Python Notebooks, Pipelines, and Datalake toolsets. We’ll integrate these tools with a custom Identity Database and an AI Assistant. Our mission? To explore the core AI functionalities that set modern applications apart—think embeddings, semantic kernel, and vectors. As we navigate Microsoft Azure’s vast offerings, we’ll build our solution from scratch...
Prerequisites for Workshop
Apart from this guide, everything will be shared through GitHub; nevertheless we need:

Azure Subscription, access to Azure OpenAI with text-embeddings ad chat-gpt deployments, Microsoft Fabric with a Pro license (trial is fine), patience and excitement!
Infrastructure

I do respect everyone’s time and i am going to point you to the Git Hub repo that holds the whole implementation, along with Terraform automation. We will start with the SQL query that is running within terraform. The query needs the following code:

CREATE TABLE Users (
    UserId INT IDENTITY(1,1) PRIMARY KEY,
    FirstName NVARCHAR(50) NOT NULL,
    LastName NVARCHAR(50) NOT NULL,
    Username NVARCHAR(50) UNIQUE NOT NULL,
    PasswordHash NVARCHAR(255) NOT NULL,
    Age INT NOT NULL,
    photoUrl NVARCHAR(500) NOT NULL
);

-- Genres table
CREATE TABLE Genres (
    GenreId INT PRIMARY KEY IDENTITY(1,1),
    GenreName NVARCHAR(50)
);

-- UsersGenres join table
CREATE TABLE UsersGenres (
    UserId INT,
    GenreId INT,
    FOREIGN KEY (UserId) REFERENCES Users(UserId),
    FOREIGN KEY (GenreId) REFERENCES Genres(GenreId)
);
ALTER DATABASE usersdb01  
SET CHANGE_TRACKING = ON  
(CHANGE_RETENTION = 2 DAYS, AUTO_CLEANUP = ON)

We have enabled Change Tracking in case we wan to trigger the Embeddings creation upon each change on the Database.

You can see we are using a JOIN statement to handle users and genres since the various genres selected by the users will help the assistant to make recommendations. We are also enabling Change Tracking so we can trigger updates for the Vector once a change is made. Keep in mind you need the sqlcmd installed on your Workstation !

Also you must add the OpenAI and AI Search Endpoint & API KEYs to Azure Key Vault with the names declared in the backend. You have to assign yourself the Key Vault Administrator role for that.
Vector, Embeddings & Fabric Pipelines

Yes you read it well! We are going to get a Books Dataset from Kaggle, clean it , transform it and upload it to AI Search, where we will create an index for the books. We will also create and store the embeddings using Vector Profile from AI Search. In a similar manner we will get the Users from SQL and upload them to AI Search users index, create the embeddings and save them as well. The real exciting stuff is that we will use Microsoft Fabric Pipelines and Notebooks for the books index and embeddings ! So it is important to have a Fabric Pro Trial License with the minimum capacity enabled.
Books Dataset

The ultimate purpose here is to achieve automation for the creation of embeddings for both Books and Users datasets, so on the Web App we can get recommendations based on preferences but also on actual queries we set to the AI Assistant. We will get a main books dataset as Delimited Text (CSV) and transform it to JSON with correct format so it can be uploaded to Azure AI Search index, utilizing the native AI Search vector profiles and Azure OpenAI for the embeddings. The Fabric Pipelines will be triggered on schedule and we will explore other possible ways.

In Microsoft Fabric, Notebooks are an important tool as in most modern Data Platforms. The managed Spark Clusters allows us to create and execute powerful scripts in the form of Python Notebooks (PySpark), add them in a Pipeline and build solid Projects and Solutions. Microsoft Fabric provides the ability to pre install libraries and configure our Spark Compute within Environments, so our code will have all requirements in this managed environment. In our case we will install all required libraries and also pin the OpenAI version to pre 1.0.0 for this project. But let’s take it from the start. We need to access app.fabric.microsoft.com and create a new Workspace with a Trial Pro License. It should look like this and also has the diamond icon:

Once we have our Workspace in place we can select it and from the left menu select New and create the Environment and later a Lakehouse.

The Environment settings that worked for me are the following, you can see that we just install Public Libraries:

Since all the code will be available on GitHub i prefer to explore the next task, create the Pipeline, which will contain the Notebooks. Select your Workspace icon on the left vertical menu, find the NEW+ drop-down menu and More Options until you find the Data Pipeline. You will be presented with the familiar Synapse\Data Factory dashboard (quite similar) where we can start inserting our activities. You have to create all Notebooks before hands just to keep everything in order. So based on the GitHub we will have 5 Notebooks ready. The Fabric API does not support yet firing pipelines, it will happen eventually, so can either schedule or work with Event Stream. The Reflex supports same Directory Azure Connections only ( We will have a look another time), but our Subscription is on another Tenant so yeah! Schedule it is !

The Pipeline has the following activities:

Let’s shed some light !

Once we have a Books Dataset (search Kaggle) we upload it to Azure Blob Storage. Then all yu have to do is to create a Copy Task from Fabric, to get the file fro Blob Storage into the Lakehouse, Pretty simple and straightforward.

We assume that the Dataset is stored in Blob Storage Account so we get that CSV into the Lakehouse. First Notebook is cleaning the data with Python, remove nulls, remove non-English characters and so on. Since the activity stores it as part of a Folder-like structure with non-direct access we need a task to save it on our Lakehouse. We then transform to JSON, make the JSON a correct array set of records, again save it to Lakehouse and the last 2 Notebooks are creating the AI Search Index, uploading the JSON to AI Search, configure the AI Search with vector and semantic profiles and get all records to create embeddings from Azure OpenAI and store those back to AI Search. Due to the great number of Documents we apply rate-limit evasion (back-off) and you can be sure this will take almost 30 minutes to conclude for around 9500 records.
Users Dataset

Most of the workflow is similar for the users index and embeddings. The difference is that our users are stored and updated with new ones, in an Azure SQL Database. Since we utilize pipelines, Microsoft Fabric natively connects to Azure SQL and in fact our activity is a Copy Task but we have a query to bring SQL data.

    SELECT u.UserId, u.Age, STRING_AGG(g.GenreName, ',') AS Genres
    FROM Users u
    JOIN UsersGenres ug ON u.UserId = ug.UserId
    JOIN Genres g ON ug.GenreId = g.GenreId
    GROUP BY u.UserId, u.Age

This SQL query is selecting data from three related tables: Users, UsersGenres, and Genres. Specifically, it’s returning a list of users (based on their UserId and Age) along with a comma-separated list of all the genres associated with each user. The STRING_AGG function is used to concatenate the GenreName into a single string, separated by commas. The JOIN operations are used to link the tables together based on common fields – in this case, the UserId in the Users and UsersGenres tables, and the GenreId in the UsersGenres and Genres tables. The GROUP BY clause is grouping the results by both UserId and Age, meaning that each row in the output will represent a unique combination of these two fields.
So it is a simpler process after all, and due to the small amount of users ( i can only subscribe up to 5-6 imaginary accounts ! ), it is a quicker process.

So what have we done so far ? Well let’s break it down, shall we ?
Process

Created the main Infrastructure using Terraform – available on GitHub
The Infra provides a Web UI where we register as users and select favorite book Genres, and can login into a Dashboard that we have access to an AI Assistant. The database used to store User’s info is Azure SQL. The Infrastructure consists also of Azure Key Vault, Azure Container Registry, Azure AI Search and Azure Web Apps. A separate Azure OpenAI is already in place.
The backend creates a Join Table to store UserId with Genres so later it will be easier to create personalized recommendations
We got a Books dataset with [id, Author, Title, Genres, Rating] fields and upload it to Azure Blob Storage
We activated Trial (or just have available) license for Microsoft Fabric capacity
We created Jupyter Notebooks to clean the source books dataset, transform it and store it as JSON
We created a Fabric Pipeline integrating these Notebooks and new ones that create a books-index in Azure AI Search, configure it with Vector and Semantic Profiles and uploaded all JSON records in it
The Pipeline continues with additional Notebooks that create embeddings with Azure OpenAI and store this embeddings back in Azure AI Search.
A new Pipeline has been deployed, that gets the Users data with a query that combines the Genres information with Users from the Azure SQL Database resource and stores it as JSON
The users Pipeline creates and configures a new users-index in Azure AI Search, configures Vector and Semantic profiles and creates embeddings, for all data, with Azure OpenAI and stores the embeddings back to the index.

Now we are left with the Backend details and maybe some minor changes for the Frontend. As you will see the GitHub repo contains all required files to create a Docker Image, push it to Container Registry and create a Web App in Azure Web Apps. Use: [ docker build -t backend . ] and tag and push: [ docker tag backend {acrname}.azurecr.io/backend:v1 ] , [ docker push {acrname}.azurecr.io/backend:v1 ]. We will be able to see our new Repo on Azure Container Registry and deploy our new Web App :

Don’t forget to add * in CORS settings for the backend Web App!

The overall Architecture is like this:

The only variable needed for the Backend Web App is the KeyVault name and the User Assigned Managed Identity ID. All access to other services (SQL, Storage Account, Ai Search, Azure OpenAI) is going through Key Vault Secrets.

Let’s have a quick look on our Backend:

import dotenv from 'dotenv';
import express from 'express';
import sql from 'mssql';
import bcrypt from 'bcrypt';
import jwt from 'jsonwebtoken';
import multer from 'multer';
import azureStorage from 'azure-storage';
import getStream from 'into-stream';
import cors from 'cors';
import { SecretClient } from "@azure/keyvault-secrets";
import { DefaultAzureCredential } from "@azure/identity";
import { OpenAIClient, AzureKeyCredential } from '@azure/openai';
import { SearchClient } from '@azure/search-documents';
import bodyParser from 'body-parser';

dotenv.config();

const app = express();
app.use(cors({ origin: '*' }));
app.use((req, res, next) => {
    res.setHeader('X-Content-Type-Options', 'nosniff');
    next();
});

app.use(express.json());
// set up rate limiter: maximum of five requests per minute
var RateLimit = require('express-rate-limit');
var limiter = RateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // max 100 requests per windowMs
});

// apply rate limiter to all requests
app.use(limiter);

app.get('/:path', function(req, res) {
  let path = req.params.path;
  if (isValidPath(path))
    res.sendFile(path);
});
const vaultName = process.env.AZURE_KEY_VAULT_NAME;
const vaultUrl = `https://${vaultName}.vault.azure.net`;
const credential = new DefaultAzureCredential({
    managedIdentityClientId: process.env.MANAGED_IDENTITY_CLIENT_ID, // Use environment variable for managed identity client ID
});
const secretClient = new SecretClient(vaultUrl, credential);

async function getSecret(secretName) {
    const secret = await secretClient.getSecret(secretName);
    return secret.value;
}

const inMemoryStorage = multer.memoryStorage();
const uploadStrategy = multer({ storage: inMemoryStorage }).single('photo');

let sqlConfig;
let storageAccountName;
let azureStorageConnectionString;
let jwtSecret;
let searchEndpoint;
let searchApiKey;
let openaiEndpoint;
let openaiApiKey;

async function initializeApp() {
    sqlConfig = {
        user: await getSecret("sql-admin-username"),
        password: await getSecret("sql-admin-password"),
        database: await getSecret("sql-database-name"),
        server: await getSecret("sql-server-name"),
        options: {
            encrypt: true,
            trustServerCertificate: false
        }
    };

    storageAccountName = await getSecret("storage-account-name");
    azureStorageConnectionString = await getSecret("storage-account-connection-string");
    jwtSecret = await getSecret("jwt-secret");
    searchEndpoint = await getSecret("search-endpoint");
    searchApiKey = await getSecret("search-apikey");
    openaiEndpoint = await getSecret("openai-endpoint");
    openaiApiKey = await getSecret("openai-apikey");

    //console.log("SQL Config:", sqlConfig);
    // console.log("Storage Account Name:", storageAccountName);
    // console.log("Azure Storage Connection String:", azureStorageConnectionString);
    // console.log("JWT Secret:", jwtSecret);
    // console.log("Search Endpoint:", searchEndpoint);
    // console.log("Search API Key:", searchApiKey);
    // console.log("OpenAI Endpoint:", openaiEndpoint);
    // console.log("OpenAI API Key:", openaiApiKey);

    // Initialize OpenAI and Azure Search clients
    const openaiClient = new OpenAIClient(openaiEndpoint, new AzureKeyCredential(openaiApiKey));
    const userSearchClient = new SearchClient(searchEndpoint, 'users-index', new AzureKeyCredential(searchApiKey));
    const bookSearchClient = new SearchClient(searchEndpoint, 'books-index', new AzureKeyCredential(searchApiKey));

    // Start server
    const PORT = process.env.PORT || 3001;
    app.listen(PORT, () => {
        console.log(`Server is running on port ${PORT}`);
    }).on('error', error => {
        console.error("Error initializing application:", error);
    });
}
initializeApp().catch(error => {
    console.error("Error initializing application:", error);
});

// Upload photo endpoint
app.post('/uploadphoto', uploadStrategy, (req, res) => {
    if (!req.file) {
        return res.status(400).send('No file uploaded.');
    }

    const blobName = `userphotos/${Date.now()}_${req.file.originalname}`;
    const stream = getStream(req.file.buffer);
    const streamLength = req.file.buffer.length;
    const blobService = azureStorage.createBlobService(azureStorageConnectionString);

    blobService.createBlockBlobFromStream('pics', blobName, stream, streamLength, err => {
        if (err) {
            console.error(err);
            res.status(500).send('Error uploading the file');
        } else {
            const photoUrl = `https://${storageAccountName}.blob.core.windows.net/pics/${blobName}`;
            res.status(200).send({ photoUrl });
        }
    });
});

// Register endpoint
app.post('/register', uploadStrategy, async (req, res) => {
    const { firstName, lastName, username, password, age, emailAddress, genres } = req.body;
    if (!password) {
        return res.status(400).send({ message: 'Password is required' });
    }

    let photoUrl = '';
    if (req.file) {
        const blobName = `userphotos/${Date.now()}_${req.file.originalname}`;
        const stream = getStream(req.file.buffer);
        const streamLength = req.file.buffer.length;
        const blobService = azureStorage.createBlobService(azureStorageConnectionString);

        await new Promise((resolve, reject) => {
            blobService.createBlockBlobFromStream('pics', blobName, stream, streamLength, err => {
                if (err) {
                    console.error(err);
                    reject(err);
                } else {
                    photoUrl = `https://${storageAccountName}.blob.core.windows.net/pics/${blobName}`;
                    resolve();
                }
            });
        });
    }

    const hashedPassword = await bcrypt.hash(password, 10);

    try {
        let pool = await sql.connect(sqlConfig);
        let result = await pool.request()
            .input('username', sql.NVarChar, username)
            .input('password', sql.NVarChar, hashedPassword)
            .input('firstname', sql.NVarChar, firstName)
            .input('lastname', sql.NVarChar, lastName)
            .input('age', sql.Int, age)
            .input('emailAddress', sql.NVarChar, emailAddress)
            .input('photoUrl', sql.NVarChar, photoUrl)
            .query(`
                INSERT INTO Users 
                (Username, PasswordHash, FirstName, LastName, Age, EmailAddress, PhotoUrl) 
                VALUES 
                (@username, @password, @firstname, @lastname, @age, @emailAddress, @photoUrl);
                SELECT SCOPE_IDENTITY() AS UserId;
            `);

        const userId = result.recordset[0].UserId;

        if (genres && genres.length > 0) {
            const genreNames = genres.split(','); // Assuming genres are sent as a comma-separated string
            for (const genreName of genreNames) {
                let genreResult = await pool.request()
                    .input('genreName', sql.NVarChar, genreName.trim())
                    .query(`
                        IF NOT EXISTS (SELECT 1 FROM Genres WHERE GenreName = @genreName)
                        BEGIN
                            INSERT INTO Genres (GenreName) VALUES (@genreName);
                        END
                        SELECT GenreId FROM Genres WHERE GenreName = @genreName;
                    `);

                const genreId = genreResult.recordset[0].GenreId;

                await pool.request()
                    .input('userId', sql.Int, userId)
                    .input('genreId', sql.Int, genreId)
                    .query('INSERT INTO UsersGenres (UserId, GenreId) VALUES (@userId, @genreId)');
            }
        }

        res.status(201).send({ message: 'User registered successfully' });
    } catch (error) {
        console.error(error);
        res.status(500).send({ message: 'Error registering user' });
    }
});

// Login endpoint
app.post('/login', async (req, res) => {
    try {
        let pool = await sql.connect(sqlConfig);
        let result = await pool.request()
            .input('username', sql.NVarChar, req.body.username)
            .query('SELECT UserId, PasswordHash FROM Users WHERE Username = @username');

        if (result.recordset.length === 0) {
            return res.status(401).send({ message: 'Invalid username or password' });
        }

        const user = result.recordset[0];
        const validPassword = await bcrypt.compare(req.body.password, user.PasswordHash);

        if (!validPassword) {
            return res.status(401).send({ message: 'Invalid username or password' });
        }

        const token = jwt.sign({ UserId: user.UserId }, jwtSecret, { expiresIn: '1h' });
        res.send({ token: token, UserId: user.UserId });
    } catch (error) {
        console.error(error);
        res.status(500).send({ message: 'Error logging in' });
    }
});

// Get user data endpoint
app.get('/user/:UserId', async (req, res) => {
    try {
        let pool = await sql.connect(sqlConfig);
        let result = await pool.request()
            .input('UserId', sql.Int, req.params.UserId)
            .query('SELECT Username, FirstName, LastName, Age, EmailAddress, PhotoUrl FROM Users WHERE UserId = @UserId');

        if (result.recordset.length === 0) {
            return res.status(404).send({ message: 'User not found' });
        }

        const user = result.recordset[0];
        res.send(user);
    } catch (error) {
        console.error(error);
        res.status(500).send({ message: 'Error fetching user data' });
    }
});

// AI Assistant endpoint for book questions and recommendations
app.post('/ai-assistant', async (req, res) => {
    const { query, userId } = req.body;

    console.log('Received request body:', req.body);
    console.log('Extracted userId:', userId);

    try {
        if (!userId) {
            console.error('User ID is missing from the request.');
            return res.status(400).send({ message: 'User ID is required.' });
        }

        //console.log(`Received request for user ID: ${userId}`);

        // Retrieve user data
        let pool = await sql.connect(sqlConfig);
        let userResult = await pool.request()
            .input('UserId', sql.Int, userId)
            .query('SELECT * FROM Users WHERE UserId = @UserId');

        const user = userResult.recordset[0];

        if (!user) {
            console.error(`User with ID ${userId} not found.`);
            return res.status(404).send({ message: `User with ID ${userId} not found.` });
        }

        console.log(`User data: ${JSON.stringify(user)}`);

        if (query.toLowerCase().includes("recommendation")) {
            // Fetch user genres
            const userGenresResult = await pool.request()
                .input('UserId', sql.Int, userId)
                .query('SELECT GenreName FROM Genres g JOIN UsersGenres ug ON g.GenreId = ug.GenreId WHERE ug.UserId = @UserId');

            const userGenres = userGenresResult.recordset.map(record => record.GenreName).join(' ');

            //console.log(`User genres: ${userGenres}`);

            // Fetch user embedding from search index
            const userSearchClient = new SearchClient(searchEndpoint, 'users-index', new AzureKeyCredential(searchApiKey));
            const userEmbeddingResult = await userSearchClient.getDocument(String(user.UserId));
            const userEmbedding = userEmbeddingResult.Embedding;

            //console.log(`User embedding result: ${JSON.stringify(userEmbeddingResult)}`);
            //console.log(`User embedding: ${userEmbedding}`);

            if (!userEmbedding || userEmbedding.length === 0) {
                console.error('User embedding not found.');
                return res.status(500).send({ message: 'User embedding not found.' });
            }

            // Search for recommendations
            const bookSearchClient = new SearchClient(searchEndpoint, 'books-index', new AzureKeyCredential(searchApiKey));
            const searchResponse = await bookSearchClient.search("*", {
                vectors: [{
                    value: userEmbedding,
                    fields: ["Embedding"],
                    kNearestNeighborsCount: 5
                }],
                includeTotalCount: true,
                select: ["Title", "Author"]
            });

            const recommendations = [];
            for await (const result of searchResponse.results) {
                recommendations.push({
                    title: result.document.Title,
                    author: result.document.Author,
                    score: result.score
                });
            }

            // Limit recommendations to top 5
            const topRecommendations = recommendations.slice(0, 5);

            return res.json({ response: "Here are some personalized recommendations for you:", recommendations: topRecommendations });
        } else {
            // General book query
            const openaiClient = new OpenAIClient(openaiEndpoint, new AzureKeyCredential(openaiApiKey));
            const deploymentId = "gpt";  // Replace with your deployment ID

            // Extract rating and genre from query
            const ratingMatch = query.match(/rating over (\d+(\.\d+)?)/);
            const genreMatch = query.match(/genre (\w+)/i);
            const rating = ratingMatch ? parseFloat(ratingMatch[1]) : null;
            const genre = genreMatch ? genreMatch[1] : null;

            if (rating && genre) {
                // Search for books with the specified genre and rating
                const bookSearchClient = new SearchClient(searchEndpoint, 'books-index', new AzureKeyCredential(searchApiKey));
                const searchResponse = await bookSearchClient.search("*", {
                    filter: `Rating gt ${rating} and Genres/any(g: g eq '${genre}')`,
                    top: 5,
                    select: ["Title", "Author", "Rating"]
                });

                const books = [];
                for await (const result of searchResponse.results) {
                    books.push({
                        title: result.document.Title,
                        author: result.document.Author,
                        rating: result.document.Rating
                    });
                }

                const bookResponse = books.map(book => `${book.title} by ${book.author} with rating ${book.rating}`).join('\n');
                return res.json({ response: `Here are 5 books with rating over ${rating} in ${genre} genre:\n${bookResponse}` });
            } else {
                // Handle general queries about books using OpenAI with streaming chat completions
                const events = await openaiClient.streamChatCompletions(
                    deploymentId,
                    [
                        { role: "system", content: "You are a helpful assistant that answers questions about books and provides personalized recommendations." },
                        { role: "user", content: query }
                    ],
                    { maxTokens: 350 }
                );

                let aiResponse = "";
                for await (const event of events) {
                    for (const choice of event.choices) {
                        aiResponse += choice.delta?.content || '';
                    }
                }

                return res.json({ response: aiResponse });
            }
        }
    } catch (error) {
        console.error('Error processing AI Assistant request:', error);
        return res.status(500).send({ message: 'Error processing your request.' });
    }
});

As you can see apart form the registration and login endpoints we have the ai-assistant endpoint. Users are able not only to get personalized recommendations when the word “recommendations” is in the chat, but also information on Genres and ratings, again when these words are in the Chat request. Also they can chat regularly with the Assistant about books and literature!

Conclusion

We just build our own Web AI Assistant with an enhanced recommendation engine, utilizing a number of Azure and Microsoft Services. It is important to prepare well ahead of such a project, load yourself with patience and be prepared to make mistakes and learn ! I reached 15 Docker Images for the backend to have a basic functionality ! But hey i did it for everyone so you can just grab it and enjoy it, even make it better! Thank you for staying up to this point!
Code and Notebooks : GitHub
References

Azure SDK for JavaScript
Azure AI Search
Create a Vector Index
Generate Embeddings
Fabric: Introduction to deployment pipelines
Develop, execute, and manage Microsoft Fabric notebooks
This blog post is initially published on CloudBlogger