DEV Community: Harsh Trivedi

Solving the Background Verification Bottleneck for enterprises: Inside BGVerifyIQ

Harsh Trivedi — Thu, 03 Sep 2026 03:20:11 +0000

Imagine an HR manager who has just completed this month’s hiring drive. Dozens of candidates have accepted their offers, and everyone is eager to onboard them as quickly as possible. But before that can happen, every background verification request has to move through multiple checks, vendors, approvals, and status updates. What should be a straightforward operational process often turns into hours of chasing spreadsheets, checking vendor portals, and answering the same status questions over and over again. It isn’t a technology problem as much as it is a visibility problem — and that’s the challenge that inspired BGVerifyIQ.

The problem every HR team quietly struggles with

Ask any HR or Talent Acquisition team what happens after an offer letter goes out, and you’ll hear a familiar story. A candidate is selected. A background verification (BGV) request is raised with a vendor. And then… the waiting begins.

Behind that wait is a surprising amount of manual work:

HR has to log into multiple vendor portals just to check where a case stands.
Status updates live in scattered Excel sheets, emails, and vendor dashboards that rarely agree with each other.
Nobody has a single, live number for “how many candidates are actually ready to be approved today.”
SLA breaches — cases that have been open too long — get discovered late, usually when a hiring manager complains.
When a discrepancy or a “suspect” flag shows up, it can get buried in a spreadsheet row instead of being escalated immediately.
Every time leadership asks “how are our BGV vendors performing,” someone spends half a day pulling numbers together manually.
New joiners get onboarded before their verification is even reviewed properly, simply because nobody had a clear, current view of their case.

None of this is because HR teams aren’t diligent. It’s because the tools weren’t built for this specific job. Spreadsheets don’t talk to vendor systems. Vendor portals don’t talk to each other. And nobody has time to become a SQL expert just to answer a simple question like “which candidates are stuck.”

This is the exact gap BGVerifyIQ was built to close.

What BGVerifyIQ actually does

BGVerifyIQ is a single workspace that sits on top of your background verification data — requests, checks, education records, employment records — and gives HR three things they didn’t have before, all in one place:

A plain-English way to ask questions, without writing a single line of SQL.
A live, always-current dashboard of the entire BGV pipeline — no manual pulling of numbers.
A complete 360° view of any single candidate, with a clear, explainable recommendation on whether they’re ready to be approved.
Everything in the current version is intentionally read-only. It never changes your underlying data. It only helps people see it, understand it, and act on it faster. That’s an important design decision we’ll come back to later.

Why I didn’t just build “another chatbot”

There’s a lot of excitement right now about AI agents that can answer any question in a chat window. And that excitement is well placed — for the right kind of question.

But here’s the catch: a chat window is great when you have a specific question (“what’s the status of candidate Ananya?”). It’s a poor way to see the big picture (“how many candidates are pending across all vendors right now, broken down by how old each case is?”). Nobody wants to type ten different questions into a chatbot every morning just to get their daily overview.

*So instead of picking one or the other, BGVerifyIQ combines both, deliberately, as separate pages built for separate jobs:
*

Chat — for quick, specific, in-the-moment questions.
KPI Dashboard — for the daily/weekly bird’s-eye view, the numbers leadership actually cares about.
Candidate 360 — for a deep dive into one candidate before making an approval decision.
History — because past answers and past conversations shouldn’t disappear the moment you close the tab.
Settings — so each user can control their own experience and see exactly which AI models are answering their questions.

This “agent + dashboard” hybrid is, in our view, the right shape for most internal business tools going forward. The agent handles the ad hoc and the unexpected. The dashboard handles the routine and the recurring. Neither replaces the other.

A tour of the app

💬** Chat**

This is where a user can type something like “show me approval-ready candidates” or “any updates on Rao” in completely normal English. Behind the scenes, the question is turned into a safe, read-only database query, run against the data, and answered with a short summary plus the actual table of results underneath — so nothing is ever “just trust the AI,” the numbers are right there to verify.

Three quick-action buttons — Priority Queue, SLA Breaches, Vendor Performance — skip the AI translation step entirely and jump straight to pre-built, always-correct answers for the questions people ask most often. Every question and every answer is automatically saved, so nothing is lost.

📊 KPI Dashboard

This is the “morning coffee” page — a live, numbers-first view of the entire BGV pipeline: total requests, how many are cleared, how many are pending, and how many have a major discrepancy. It also shows a status breakdown, how long cases have been open (aging buckets), how each vendor is performing on turnaround time, and — critically — a live list of every case currently breaching its SLA.

At the bottom sits the Priority Queue: a ranked list of exactly which candidates need attention today, downloadable as a CSV for anyone who wants to share it further.

👤 Candidate 360

Search for any candidate by name or ID, and this page pulls together everything known about them into one screen: overall status, an approval-readiness badge, a risk score, the assigned vendor and vendor contact, a check-by-check breakdown (education, employment, address, criminal, identity), a dated timeline of everything that’s happened on the case, and — importantly — a plain-English explanation of why the system is recommending “Ready,” “Caution,” “Needs Review,” or “Escalate.” This isn’t a black-box score; every recommendation comes with the specific reasons behind it, so HR can trust it and defend the decision if asked.

From here, HR can download a one-page PDF report or email it directly to a hiring manager or stakeholder.

🕘 History

Every past conversation from the Chat page, tied to the logged-in user’s email, searchable by keyword, newest first. Click “Open” on any past conversation and it reloads right back into Chat so the person can keep going from where they left off.

⚙️ Settings

Users can update their name, role, and email, switch between light and dark theme, and — importantly for trust and transparency — see exactly which AI model is currently answering their questions, and confirm that the system is running in read-only mode.

About
This section explains what each page is about and what BGVerifyIQ stands for.

Dark Mode

What’s happening behind the scenes

For the developers and technical stakeholders reading this, here’s a quick look at how the current version is built, and why each piece exists.
db.py — handles the connection to the PostgreSQL database. It reads connection details from environment variables (so nothing sensitive is hard-coded), keeps one reusable connection pool for performance, and includes a small but important safety function: is_read_only_sql(). This function checks every single query before it's allowed to run, and blocks anything that isn't a plain SELECT. It's the guardrail that makes it safe to let an AI generate SQL on the fly — even if the AI ever tried to write something destructive, this layer would refuse it.

llm.py — the AI layer that turns a plain-English question into a safe SQL query, and turns raw query results back into a clear, professional summary. It's built so that if one AI provider is slow or down, it automatically falls back to a second one — so the app doesn't just stop working during a demo or a busy morning. It also carries strict rules about how to match candidate names and IDs (using flexible partial matching instead of exact matches, since people rarely type a name or ID perfectly), which is a big part of why the Chat page actually finds what users are looking for instead of returning "no results."

pdf_report.py — generates the one-page Candidate 360 PDF on demand, entirely in memory, with careful handling so that special characters (like smart quotes or long names) never break the PDF layout.

mailer.py — sends the PDF report directly to a stakeholder's inbox. Worth noting: this file also tells a useful real-world lesson. It was originally built using standard SMTP email sending, which worked perfectly on a local machine, but failed once deployed to a cloud platform — because most cloud hosting providers block outbound SMTP traffic to prevent spam abuse. The fix was to switch to a transactional email API (Brevo) that works over regular HTTPS, which is never blocked. It's a small example of the kind of real infrastructure problem that only shows up once you actually deploy something.

The data layer — a clean PostgreSQL schema (bgv) with tables for requests, checks, education, and employment history, plus three pre-built views (v_candidate_360, v_sla_breaches, v_vendor_performance) that do the heavy lifting of joining and aggregating data. The AI is instructed to always prefer these views over writing its own joins — which keeps its answers consistent and correct.

Try it yourself

A live version of BGVerifyIQ is deployed and ready to explore:

bgverifyiq-production.up.railway.app

Feel free to click through every page — Chat, KPI Dashboard, Candidate 360, History, and Settings — with sample data already loaded, so you can see exactly how it behaves before considering it for your own team.

From prototype to enterprise: what a real rollout looks like

What’s described above is a working, functional prototype — and a genuinely useful one. But there’s an honest and important distinction between “a tool that solves the problem well for a demo or a small team” and “a tool that’s ready to run background verification for thousands of employees across an entire organization.” Here’s what that jump actually involves.

A production-grade frontend
The current interface is built using Streamlit, which is excellent for proving out an idea fast. For an organization-wide rollout, this would typically move to a proper frontend framework — Angular, React, or similar — giving finer control over performance, accessibility, custom branding, and integration with existing internal design systems.

A real backend, with security built in from day one
An enterprise version needs a proper backend service layer, not just a script talking to a database. That means:

Authentication — logging in through the company’s existing identity provider (SSO/SAML/Azure AD), not a name-and-email form.
Authorization — making sure a regional HR analyst can only see their region’s candidates, while a compliance officer might see everything, and a vendor SPOC sees only their own cases. Not everyone should see everyone’s data. Basically Role Based Access Control, data masking policies should be enabled. [We will discuss both RBAC and Data Masking in snowflake in another article]
Data security — encryption of data at rest and in transit, strict handling of personally identifiable information (PII), and audit logs of who viewed or changed what, and when.
Proper exception handling — every failure (a database timeout, an AI provider being slow, a malformed input) should fail gracefully with a clear message, never a broken page.
Testing — automated tests for the business rules (like the risk-scoring and readiness logic), so a future code change can’t silently break how a candidate gets flagged.
A feedback mechanism — a simple way for HR users to flag when an AI-generated answer was wrong or unhelpful, so the system keeps improving based on real usage instead of guesswork.

Using AI models your organization already trusts and governs

This is one of the most important points for any organization considering this seriously.

The current prototype uses free, hosted, general-purpose AI models (Groq’s Llama model, and Google’s Gemini, as a backup). These are great for building and proving out an idea quickly — but they were not built or contracted for handling real, sensitive employee data such as names, ID numbers, addresses, and background check outcomes. An organization should not send real employee records to a public, general-purpose AI API unless that API is specifically covered by the company’s data processing agreements and security review.

The right approach for a real deployment is to use whatever AI platform your organization has already vetted and governs — most companies today are standardized on either Snowflake or Databricks for their data warehouse, and both now offer their own governed AI agents:

If your organization’s data lives in Snowflake, you can use Snowflake Cortex Agents to run directly on your existing tables and views — the exact same bgv schema and views shown above would work almost as-is. The AI never leaves Snowflake's security boundary, meaning your data doesn't get sent to an outside company's servers at all.
If your organization uses Databricks, the same idea applies with Databricks’ Mosaic AI agents running on your Lakehouse tables.

Beyond just being safer, these governed platforms typically unlock more capability too — automatically generating charts, producing longer comparative analyses (“compare this quarter’s vendor turnaround to last quarter”), and integrating with the rest of your company’s existing data and security tooling, rather than living as a separate, isolated app.

In short: build and prove the idea with free/open tools first (as I did here), then swap in your organization’s approved, governed AI platform before any real employee data touches it.

Additional functionality worth adding

Once the security and governance foundation is in place, a few natural next features come to mind:

A CRUD page to approve or update candidate status directly from the app — instead of just viewing data, an authorized HR user could mark a candidate as approved, add a remark, or update a case status, with that change written straight back to the source system, fully audit-logged.
Notifications — automatic alerts (email, Slack, Teams) the moment a case breaches SLA or a suspect flag appears, instead of HR having to go looking for it.
Integration with the HRMS/ATS — so a “Ready” status in BGVerifyIQ can automatically trigger the next onboarding step, instead of someone manually copying that status elsewhere.
Feedback and continuous improvement loop — letting users rate AI answers, which helps refine the system over time.

Why an AI agent at all — and why it matters even more on mobile

It’s worth pausing on this, because “just add an AI chatbot” can sound like a trend rather than a real solution. Here’s the actual problem an agent solves.

Traditionally, if an HR analyst wanted to answer a slightly unusual question — “how many campus hires are stuck with a discrepancy this month?” — they had one of two options: ask someone technical to write a query for them, or manually filter and cross-reference spreadsheets by hand. Both are slow, and both mean the person asking the question depends on someone else, or on their own patience with Excel.

An AI agent removes that dependency. It lets someone simply ask, in their own words, and get a trustworthy, data-backed answer in seconds — without needing to know what a database even is. That’s the real value: it turns “I need to go find someone who can pull this” into “I just asked, and I have my answer.”

This matters even more on mobile. A dashboard with charts and tables is genuinely hard to use well on a small phone screen — things get cramped, and scrolling through filters is painful. A chat interface, on the other hand, is something almost everyone is already completely comfortable with, because it looks and feels exactly like every messaging app they already use. On a phone, “just type your question” is a far better experience than “pinch and zoom into a dashboard.” This is a big part of why the agent and the dashboard are built as complementary tools rather than the dashboard being the only option — mobile users, in particular, benefit enormously from being able to just ask.

What this actually solves — the short version

For organization leaders and executives, here’s the summary:

Saves time — no more logging into multiple vendor portals or manually cross-referencing spreadsheets to answer a simple status question.
Faster approvals — a clear, explainable readiness badge tells HR exactly who’s ready to move forward, and why, without second-guessing.
Fewer surprises — SLA breaches and risk flags are surfaced automatically and immediately, instead of being discovered weeks later.
Less repetitive manual work — one-click PDF reports and direct email delivery replace manually compiling updates for hiring managers.
Better decisions, not just faster reports — the KPI dashboard turns raw data into visible bottlenecks and vendor performance trends that leadership can actually act on.
A tool people will actually use — because it meets people where they already are, whether that’s a quick chat question on a phone or a full dashboard review on a desktop.

And to be clear about where this stands today: this is proof that the idea works, built on real, functioning logic — the business rules, the read-only safety design, the recommendation logic — none of that is a mockup. What comes next, for an organization ready to roll this out at scale, is wrapping that same proven logic in enterprise-grade security, governance, and infrastructure — using AI models the organization already trusts with its data, not a public API.

The underlying idea, though, doesn’t change: give HR one place to ask, one place to see, and one place to decide — instead of five different tools that don’t talk to each other.

BGVerifyIQ assists decisions — final approval always rests with the HR team.

Disclaimer

BGVerifyIQ is a demonstration project created to showcase how AI can streamline and improve the background verification process. All data, candidate names, organizations, vendors, and background verification records shown in the application are entirely fictional and generated for demonstration purposes. No real employee, customer, or organizational data has been used at any stage of development, testing, screenshots, or deployment.

The application is built on a carefully designed sample dataset that simulates realistic background verification scenarios while ensuring privacy and confidentiality.

If you have questions about the architecture, implementation, AI workflow, or would like to discuss how a similar solution could be built for your organization, I’d be happy to connect. Feel free to reach out through LinkedIn or leave a comment on this article, and I’ll do my best to help.

Securing Enterprise AI Pipelines: Using Azure Key Vault and AES Encryption for Sensitive Documents in Snowflake/Databricks AI Workloads

Harsh Trivedi — Wed, 02 Sep 2026 17:03:05 +0000

A practical architecture for securing contracts, invoices, employee data, and business documents before they are consumed by AI models, document intelligence systems, and enterprise agents.

Organizations are rapidly building AI-powered applications on top of business documents such as contracts, invoices, employee records, financial reports, sales data, and operational data. While significant effort is spent on developing AI models and intelligent agents, security is often treated as an afterthought.

One of the biggest risks in enterprise AI is exposing sensitive information during document storage, processing, and retrieval. Simply storing files in cloud storage or data platforms is not enough. The data should also be secured during transmission.

In this article, I will walk you through a security-first architecture that combines AES encryption, Azure Key Vault, Service Principals, RBAC permissions, and controlled decryption workflows to protect sensitive documents while still enabling AI-powered document extraction, analytics, and intelligent agents.
We’ll cover:

✔ The security problem enterprises face

✔ Why encrypted storage alone is not enough

✔ Why encryption keys should never be stored in code or .env files

✔ Azure Key Vault fundamentals

✔ Service Principal authentication

✔ RBAC permissions and access control

✔ Creating and managing Key Vault secrets

✔ Python implementation examples

✔ Snowflake and Databricks integration patterns

✔ Secure document processing for AI workloads

✔ Security benefits and best practices

This architecture can be applied across Snowflake, Databricks, Microsoft Fabric, Azure AI Foundry, Synapse Analytics, and custom AI platforms.

The Growing Security Challenge in Enterprise AI

Organizations are increasingly building AI applications on top of business-critical documents.

Examples include:

Customer contracts
Legal agreements
Invoices
Financial statements
Employee records
Payroll information
Vendor documents
HR reports
Employee attrition predictions
Compliance reports
Healthcare records
Procurement documents

These documents often contain highly sensitive information.

At the same time, organizations want AI systems to:

Extract information from documents
Build search experiences
Power RAG solutions
Support AI agents
Generate business insights
Answer natural language questions
Generate KPIs, charts and visuals
This creates a difficult challenge:

How can we allow AI systems to use sensitive information without exposing the underlying documents?

The Wrong Approach

Many organizations unknowingly introduce security risks by:

Storing documents unencrypted
Storing encryption keys in source code or environment files
Keeping secrets in configuration files
Saving credentials inside notebooks
Embedding keys inside ETL pipelines

For example:

AES_KEY = "MySecretEncryptionKey"

If someone gains access to:

GitHub repositories
DevOps pipelines
Virtual machines
Databases
Storage accounts they may gain access to both the encrypted documents and the encryption key.

At that point, encryption provides little value.

The Security Principle

One of the most important security principles is:

Encrypt the data and protect the encryption key separately.

Even if attackers gain access to encrypted documents, they should not be able to access the encryption key.

This is where Azure Key Vault becomes critical.

The Solution Architecture

The architecture consists of five major layers:

Document Encryption
Secure Key Storage
Controlled Authentication
In-Memory Decryption
AI Processing

Document Source — The source of document can be anything like SAP, Cloud storage, Azure Blob Storage etc.

When we move them to our storage layer, they should be encrypted.

Step-by-Step implementation guide-

A) Why We Encrypt Documents

Consider documents such as:

Client contracts
Employee compensation data
Vendor agreements
Invoices
Strategic business reports
Attrition prediction outputs These files often contain information that should not be accessible to every user or system.

Before storing these files, we encrypt them using AES-256.

This ensures that even if storage is compromised or a developer has RBAC (Role based access control), the document remains unreadable without the encryption key.

B) Creating an AES Encryption Key

Example:

from Crypto.Random import get_random_bytes
AES-256 = 32 bytes
aes_key = get_random_bytes(32)
print(aes_key.hex()) 32 bytes = 256 bits

This creates an AES-256 encryption key.

Why AES?

AES (Advanced Encryption Standard) is one of the most widely used symmetric encryption algorithms in the world.

Advantages:

Fast
Highly secure
Industry standard
Suitable for large documents
Supported across cloud platforms AES-256 is commonly used in enterprise environments.

C) What is Azure Key Vault?

Azure Key Vault is a managed Azure service designed for securely storing and controlling access to secrets, cryptographic keys, and certificates.

Instead of storing sensitive values in application code, organizations place them inside a centralized vault protected by Azure Identity and RBAC controls.

Azure Key Vault supports three primary object types:

Secrets

Examples:

API Keys
Database Passwords
OAuth Client Secrets
AES Encryption Keys

Keys

Examples:

RSA Keys
Cryptographic Signing Keys
Customer Managed Encryption Keys

Certificates

Examples:

SSL Certificates
TLS Certificates

In our use case, the AES encryption key is stored as a Secret.

Why Azure Key Vault is required?

Centralized secret management
Fine-grained access control
Secret rotation support
Access auditing
Compliance support
Reduced insider risk

Creating Azure Key Vault

1.Navigate to: portal.azure.com

Azure Portal -> Create Resource -> Key Vault -> Create

2.Configure:

Subscription

Resource Group

Region (eg. Central India)

Vault Name (eg. projectname-aes-key-kv)

3.For Authorization/Permission Model:

Select: Azure Role Based Access Control (RBAC)

4.For Networking:

Public access: Selected networks

or

Private endpoint

5.Permissions Required to Create a Key Vault

Typically one of the following roles:

Owner
Contributor
Key Vault Contributor

Important:

Key Vault Contributor can manage the Key Vault resource itself but cannot read secrets stored within the vault.

This separation improves security.

*6.Storing the AES Key in key vault secrets and the permissions that are required to create secrets.
*
IMP: When using Azure RBAC, you will need Key Vault secret officer role to create secret.

Once the role is assigned, Create and Store the generated AES key as a secret.

Example:

Secret Name:

AES-DOCUMENT-ENCRYPTION-KEY : secret _code

The key is now centrally managed and protected.

D) Creating a Service Principal Applications, requires an identity to access Azure resources.

This is where Service Principals are used.

Think of a Service Principal as:

A non-human identity used by applications.

Example:

Snowflake/Databricks/Azure Function/Fabric Pipeline/Custom API

↓

Service Principal

↓

Azure Key Vault

↓

AES Secret

The Service Principal becomes the trusted identity that accesses the vault.

Why Service Principals Are Important

Without Service Principals:

Applications would require user credentials
Automation becomes difficult
Auditing becomes harder

Service Principals provide:

Automated authentication
Secure access
RBAC integration
Least privilege access

Required: The service principal system-managed identity must be assigned read secret role on the key vault

For Snowflake, Databricks, Fabric, or AI platforms: (Your service principal name might be: projectname_aeskey_extract_spn)

Assign Role:

Key Vault Secrets User

Can:

Read secret values
Read secret metadata

E) Python Example: Reading the AES Key

from azure.identity import ClientSecretCredential
from azure.keyvault.secrets import SecretClient
credential = ClientSecretCredential(
    tenant_id="<tenant-id>",
    client_id="<client-id>",
    client_secret="<client-secret>"
)
vault_url = "https://company-kv.vault.azure.net"
client = SecretClient(
    vault_url=vault_url,
    credential=credential
)
aes_key = client.get_secret(
    "AES-DOCUMENT-ENCRYPTION-KEY"
).value

The application retrieves the key only when needed.

F) Encrypting Sensitive Documents

from Crypto.Cipher import AES
from Crypto.Util.Padding import pad
from Crypto.Random import get_random_bytes
# AES key retrieved from Azure Key Vault
key = bytes.fromhex(aes_key_hex)
# Generate random IV (16 bytes for AES)
iv = get_random_bytes(16)
with open("contract.pdf", "rb") as file:
    plaintext = file.read()
cipher = AES.new(key, AES.MODE_CBC, iv)
ciphertext = cipher.encrypt(
    pad(plaintext, AES.block_size)
)
# Store IV along with encrypted content
encrypted_content = {
    "iv": iv,
    "ciphertext": ciphertext
}

The encrypted file can now be stored safely.

G) AI Processing Pattern

When processing begins:

Retrieve AES key from Key Vault
Decrypt document in memory
Extract structured or unstructured chunk information
Persist extracted results
Discard decrypted content

The key principle is:

Decrypt only when required and only in memory.

H) Platform Integration Examples

I) AI Agents and RAG Systems

Once document information is extracted into structured datasets, AI systems can answer questions such as:

Which contracts expire next month?
Which invoices exceed $100,000?
Which vendors require renewal?
What is the predicted attrition risk for the Sales department?
Which customers have pending obligations? The AI model operates on structured business data rather than unrestricted access to sensitive raw documents.

J) Security Benefits Achieved

This architecture provides:

Encryption at Rest — Documents remain protected in storage.
Key Separation — Encryption keys are separated from encrypted data.
Least Privilege Access — Applications receive only required permissions.
Auditing — Secret access can be monitored.
Key Rotation — Encryption keys can be replaced centrally.
Compliance Support — Supports enterprise governance and security requirements.
Reduced Attack Surface — Compromised storage does not automatically expose sensitive information.

Final Thoughts

As organizations accelerate AI adoption, securing the data feeding those AI systems becomes just as important as building the models themselves.

A secure AI platform is not only about model accuracy, vector databases, or intelligent agents. It is equally about protecting contracts, invoices, employee information, and other sensitive business assets that power those systems.

By combining AES encryption, Azure Key Vault, Service Principals, RBAC permissions, and controlled in-memory decryption, organizations can build AI platforms that are secure, scalable, compliant, and enterprise-ready.

The goal is simple:

Protect the documents.
Protect the keys.
Allow AI access only when necessary.

What next?

In my next article, I will walk you through how we implemented enterprise-grade access controls for AI-powered platforms using techniques such as:

Row-Level Security (RLS)
Row Access Policies
Dynamic Data Masking
Role-Based Access Control (RBAC)
Business Unit-based filtering
Customer-based entitlements
Financial data masking
Secure document access controls
Time-limited document download links
Agent-level authorization patterns
Governance frameworks for AI applications

We’ll explore how an AI agent can return different answers to different users for the same question, while ensuring that sensitive information remains protected and compliant with organizational policies.

Because securing the documents is important.

But securing who can access the information extracted from those documents is what truly makes an AI platform enterprise-ready.