DEV Community: Mira Sloan

Why Most AI Tools Fail the Sensitive Data Test (And What Passing Actually Looks Like)

Mira Sloan — Thu, 18 Jun 2026 10:52:11 +0000

I have a test I run on every AI tool I evaluate for enterprise use. I call it the sensitive data test, and in three years of running it, most tools have failed at least part of it.

The test is simple to describe and harder to pass than it sounds.

I take a set of documents with genuinely different sensitivity levels: public company information, internal operational data, and restricted HR or financial records. I set up the tool as a normal employee would. Then I query it from three different user contexts: a general employee, a manager, and an admin. The question I am trying to answer is: can a general employee get to restricted data by asking the right question?

Most tools fail this test in the same way.

The failure mode is not that the tool has no access controls. Almost every enterprise tool has access controls. The failure mode is that the access controls live in the wrong place. They are applied at the tool interface level (this user can use the tool) or at the folder level (this user can open this folder) but not at the retrieval level (this user cannot receive this content in an AI response even if the AI has access to the underlying system).

The gap between folder-level access control and retrieval-level access control is where the sensitive data exposure happens. If the AI can retrieve a document and the user can query the AI, the user can get to the document content through the AI even if they cannot open the document directly. In traditional software this gap does not exist. In AI-powered retrieval systems it is the default.

The tools that pass the sensitive data test handle this with architectural separation rather than policy enforcement. The restricted content is not retrievable by the AI for an unauthorized user, not because a policy says "don't show this" but because the retrieval pipeline that serves that user is physically separated from the data that user is not authorized to see.

This distinction matters because policies are enforced by people following them. Architectural separation is enforced by the system regardless of configuration drift, permissions hygiene, or human error.

In practice, this means the passing tools tend to be self-hosted or private-cloud deployments where the data architecture can be designed around access control from the start. PrivOS (https://privos.ai/) is one of the tools I have evaluated that handles this at the architecture level through room-scoped isolation. The HR data is literally not in the retrieval space that a general employee AI session has access to. That is a different guarantee than "we have access controls."

The SaaS tools that pass partial versions of the test are the ones that have built retrieval-aware permission filtering, where the retrieval layer checks permissions at query time rather than filtering after the fact. This is better than nothing and worse than architectural separation. It is better because it prevents most accidental exposure. It is worse because it depends on the permission state being accurate at query time, which is only as good as your ongoing permissions maintenance.

The test I run also includes a less obvious dimension: what does the tool tell the user when it cannot answer because of access restrictions? The answer to this matters.

Tools that say nothing and return an empty result are inadvertently confirming that restricted data exists. If I ask the AI about the compensation band for a senior engineer and it returns no results, I have learned that there is probably a document about compensation bands that I am not allowed to see. That is information leakage even though no document content was disclosed.

Tools that say "I don't have information about this" are slightly better but still leaking the same signal if the user is sophisticated enough to correlate the non-answer with their expectations.

The correct answer is a response that is indistinguishable from the response the tool gives when genuinely no relevant information exists. This is hard to implement well and most tools have not tried.

The sensitive data test is worth running before deployment, not after. The results are almost always surprising and the surprises are always better to find in a test environment than in production.

Enterprise AI Pricing: What the Pricing Page Actually Means vs What You Will Actually Pay

Mira Sloan — Wed, 17 Jun 2026 15:46:01 +0000

I have helped negotiate or reviewed pricing for enterprise AI tools enough times to know that the number on the pricing page and the number on your first invoice are rarely the same. Here is the translation guide nobody gives you.

"Starting at $X per seat per month"

This is the floor, not the typical price. The per-seat number usually applies to the base tier which is missing at least one feature you specifically need. Advanced permissions, SSO, audit logs, priority support, API access above a certain rate: these are in the next tier up. Budget 1.5x to 2x the advertised per-seat price for the tier that has what enterprise actually requires.

"Usage-based pricing"

Means your bill will vary and the variance is hard to predict before deployment. Token-based pricing especially. Your pilot used careful, deliberate queries. Your production deployment with 200 employees asking casual questions will burn tokens at a completely different rate. Ask the vendor for average consumption numbers from comparable customers before you sign anything usage-based.

"Volume discounts available"

Available, not automatic. You have to ask. And the ask needs to happen before you sign, not after. I have watched procurement teams assume the discount would come up in renewal negotiations and spend 18 months paying full price because nobody brought it up at signature.

"Free tier included"

Read what is included carefully. Free tiers for enterprise AI tools almost always exclude the data handling terms you need. The free tier processes your data under consumer terms. The paid tier processes it under enterprise terms. If you are running an actual business use case, you need the paid tier regardless of whether the features would fit in the free tier.

"Annual contract with monthly billing"

You are committed for the year. The monthly billing is a cash flow convenience, not an opt-out. Some vendors will offer a genuine monthly option with a meaningful price premium. Most will offer "monthly billing" that is actually an annual contract invoiced monthly. Confirm which one you are signing.

What the pricing page does not show you

Implementation fees when they exist, which they often do for enterprise tiers. Overage charges when usage exceeds plan limits. Minimum seat counts that mean you cannot downgrade even if adoption is lower than projected. Price escalation clauses at renewal, which are common and sometimes aggressive. Data export fees when you want your own data back in a usable format.

I am not saying AI tools are bad value. Some of them are excellent value. I am saying the value calculation requires knowing the actual total number, not the marketing page number, and those two numbers are consistently different enough that it is worth the hour it takes to map out the full cost before you commit.

Ask for a sample invoice from a comparable customer. Ask what the bill would look like if usage came in 20% higher than projected. Ask what happens at renewal if you want to reduce seat count. The answers to those three questions will tell you more than anything on the pricing page.

Nobody Talked About Year Two

Mira Sloan — Tue, 16 Jun 2026 16:14:40 +0000

The first year was easy.

The pricing looked fair.

The onboarding went smoothly.

The team adopted the platform.

Everyone considered the project a success.

Then renewal season arrived.

That's when the real evaluation started.

I've seen this pattern enough times that I now pay more attention to renewal conversations than sales conversations.

Sales calls tell you how a vendor wins customers.

Renewal discussions tell you how they treat existing ones.

The First-Year Illusion

Most SaaS evaluations focus heavily on acquisition costs.

That's understandable.

It's the number buyers see first.

But first-year pricing is often the least interesting number in the entire relationship.

Because first-year pricing assumes:

• limited usage

• limited adoption

• limited dependence

A year later, none of those assumptions are true.

The software has become embedded in daily operations.

The organization now understands its value.

The vendor understands that too.

The Questions Buyers Forget To Ask

During procurement, teams usually ask:

• What's included?

• How many users are supported?

• What integrations are available?

Reasonable questions.

But I rarely hear:

• What changed for customers at renewal?

• How often do pricing structures change?

• Which features historically moved into higher tiers?

• What percentage of customers upgrade after year one?

Those answers often reveal more than the pricing page.

The Hidden Upgrade Path

One thing I've learned while reviewing SaaS platforms:

Many products don't become expensive because pricing increases.

They become expensive because organizations grow.

The team expands.

Usage expands.

Governance requirements expand.

Suddenly features that once felt optional become mandatory.

Examples include:

• audit logs

• advanced permissions

• SSO

• compliance reporting

• enterprise support

The platform didn't change.

The company's needs changed.

The invoice follows.

What I Watch For

When evaluating a vendor, I pay attention to how they discuss growth.

Do they explain future costs clearly?

Do they acknowledge scaling realities?

Do they help buyers understand the likely upgrade path?

Or do they focus exclusively on today's number?

Transparency during evaluation usually predicts transparency later.

The opposite is also true.

The Question That Matters More

I don't think buyers should focus on the cheapest platform.

I think buyers should focus on the most predictable platform.

Cheap software can become expensive.

Expensive software can become efficient.

The difference usually becomes clear only after adoption succeeds.

That's why I rarely ask:

"How much does this cost today?"

Instead, I ask:

"What will this cost when my team actually depends on it?"

In my experience, that's the question that reveals the real pricing model.

How AI SaaS Pricing Changes After You're Locked In

Mira Sloan — Mon, 15 Jun 2026 18:30:44 +0000

The first price is rarely the real price.

That's the first thing I tell founders whenever they're evaluating a new AI platform.

Not because vendors are dishonest.

Most aren't.

The problem is that the first invoice only tells part of the story.

The longer I've reviewed B2B SaaS products, the more I've noticed a pattern:

The product you buy is often not the product you end up paying for twelve months later.

And AI software seems particularly good at hiding this reality.

The Demo Price Is Usually The Cheapest Month You'll Ever Have

A vendor shows up.

The pricing looks reasonable.

The team runs a pilot.

A few users get access.

Everyone is happy.

The monthly bill feels manageable.

At this stage, the platform is competing for adoption.

Naturally, pricing feels attractive.

The challenge begins when the software becomes important.

Because once the platform becomes part of daily operations, the conversation changes.

The vendor is no longer selling software.

They're selling dependency.

Growth Creates New Costs

This is where many teams get surprised.

The platform hasn't changed.

But the company has.

Maybe the pilot involved:

10 users
one department
limited workflows
small data volumes

Six months later the reality looks different:

80 users
multiple teams
automated workflows
larger datasets
higher AI usage

Suddenly the original pricing page becomes less relevant.

The platform is solving more problems.

The bill grows accordingly.

Sometimes dramatically.

The Features You Need Later Are Rarely Included Today

One thing I always check during reviews:

What happens after adoption succeeds?

Because that's when premium features suddenly become important.

Examples include:

SSO
audit logs
role-based permissions
API access
advanced reporting
compliance controls
enterprise support

These features often aren't priorities during evaluation.

They're priorities after rollout.

That's also when organizations discover which capabilities require a higher pricing tier.

I've seen teams budget for a product and then discover six months later that the features they actually need live behind a much larger contract.

AI Usage-Based Pricing Changes The Equation

Traditional SaaS pricing was relatively predictable.

AI pricing often isn't.

Some vendors charge based on:

users
messages
tokens
compute usage
automations
workflows
storage
API requests

Sometimes several at once.

That means budgeting becomes harder.

A successful deployment may generate a larger bill than expected.

Ironically, high adoption can become the reason costs increase.

That's not necessarily bad.

But buyers should understand it before signing.

The Lock-In Phase

This is the moment I pay attention to.

Not onboarding.

Not implementation.

Lock-in.

By this stage:

data is inside the platform
workflows depend on it
employees are trained
processes have adapted

Replacing the system becomes expensive.

That's when pricing changes become more meaningful.

A small increase may no longer be a small decision.

The cost of leaving has increased.

The vendor knows this.

The buyer should know it too.

Questions I Always Ask Before Adoption

Whenever I'm evaluating a SaaS platform, I want answers to these questions:

How is pricing likely to change as usage grows?
Which features require higher tiers?
What limits exist today?
What happens if user count doubles?
What happens if AI usage triples?
Can data be exported easily?
How difficult is migration?

Notice that none of these questions focus on today's bill.

They focus on future bills.

That's where surprises usually live.

Good Vendors Make Growth Predictable

The best SaaS companies don't hide growth costs.

They explain them.

They help buyers understand:

usage drivers
scaling assumptions
upgrade paths
future requirements

That transparency builds trust.

When vendors avoid these conversations, I become cautious.

Because uncertainty is expensive.

Especially for growing organizations.

My Take

I don't think buyers should focus on the cheapest platform.

I think buyers should focus on the most predictable platform.

Cheap software can become expensive.

Expensive software can become efficient.

The difference usually becomes clear only after adoption succeeds.

That's why I rarely ask:

"How much does this cost today?"

Instead, I ask:

"What will this cost when my team actually depends on it?"

In my experience, that's the question that reveals the real pricing model.

Why General-Purpose AI Platforms Usually Win at Enterprise Scale

Mira Sloan — Fri, 12 Jun 2026 17:37:00 +0000

The conventional wisdom in enterprise software evaluation says to buy the best tool for each job. Pick the best CRM. Pick the best project management tool. Pick the best AI assistant. Use integrations to connect them.

This advice made sense when software categories were stable, integrations were reliable, and the switching costs between tools were low. None of those conditions holds for enterprise AI in 2025.

The niche AI tool, purpose-built for one specific task, optimized deeply for that task, marketed on the strength of that optimization, has a compelling pitch. And in evaluations, it usually wins on the specific task it was built for. The general-purpose platform looks like a compromise by comparison.

Over an 18-month deployment horizon, the pattern reverses. Here is why.

What Niche AI Tools Win On

Let me give the best case its due before arguing against it.

Niche AI tools win on task-specific performance. A legal contract analysis tool that has been trained on millions of contracts, fine-tuned on specific legal frameworks, and optimized for the specific question types lawyers ask will outperform a general-purpose AI platform on contract analysis tasks. This performance advantage is real, measurable, and significant in evaluations.

They also win on UX for their target workflow. A niche tool built for customer support triage looks like a tool built for customer support triage. The interface fits the workflow. The outputs are formatted for the specific downstream actions. The general-purpose platform looks like a generic tool being used for a specific purpose, because that is what it is.

These advantages are meaningful. They are also narrower than they initially appear, and they erode over time.

The Context Gap That Kills Niche Tool Value

The thing that niche AI tools cannot easily provide is organizational context.

A legal contract analysis tool knows a lot about contracts in general. It knows very little about your organization's specific negotiating positions, your preferred non-standard clauses, your historical decisions about specific risk trade-offs, the counterparties you deal with regularly, or the internal escalation logic for specific issue types.

That organizational context is where a large fraction of the value of AI-assisted contract review actually lives. The questions lawyers most need answered are not "is this a standard indemnification clause", they know that, but "given our standard negotiating position and our history with this counterparty, should we push back on this clause?"

The answer to the second question requires organizational memory, not task expertise. And organizational memory lives in the system that has access to your actual organizational data: your previous contracts, your negotiation history, your internal communications, your deal records.

A general-purpose AI platform that is integrated with the systems where organizational data lives can answer the second question. A niche tool that specializes in contract analysis but has no access to your organizational context cannot. The performance advantage on the narrow task does not compensate for the inability to answer the questions that require context.

The Integration Debt That Compounds

Every niche tool you deploy requires integration with your existing systems. The integration is rarely trivial. It involves authentication setup, data pipeline development, permission mapping, workflow design, and ongoing maintenance as both the niche tool and the integrated systems evolve.

For a single niche tool, this integration cost is manageable and may be clearly worth the performance benefit. For five niche tools, the integration cost multiplies and the integration maintenance compounds. The fifth integration is significantly harder than the first because you are now managing the interactions between five systems rather than two.

The general-purpose platform, by contrast, has one integration surface. Data flows to and from the platform through a single integration layer. Adding a new use case does not require a new integration, it requires configuring the platform for a new context within the existing integration.

Over an 18-month horizon, the organization that deployed five niche tools is managing five integrations in various states of reliability, with five vendor relationships requiring attention, five sets of security reviews, and five upgrade cycles that may create temporary incompatibilities with each other. The organization that deployed a general-purpose platform is managing one.

The Learning Concentration Problem

When your AI use cases are spread across five niche tools, the knowledge about how to use AI effectively is also spread across five isolated silos.

The customer support team learns what prompts work well in their support AI tool. That knowledge stays in the support team. The legal team learns what works in their contract AI tool. The sales team learns what works in their deal intelligence tool. None of this learning transfers.

In an organization with a general-purpose AI platform, the prompt engineering insights, the retrieval configuration learnings, the workflow design patterns, all of this accumulates in a shared knowledge base that can be applied across the organization. The support team's discovery that a specific retrieval approach improves accuracy can be applied by the legal team. The sales team's discovery that a specific output format reduces friction can be adopted everywhere.

This cross-pollination effect is not visible in any single tool evaluation but compounds significantly over time.

When Niche Tools Are Still the Right Answer

I do not want to oversell the general-purpose argument. There are scenarios where niche tools remain the better choice.

When the task-specific performance gap is genuinely material and cannot be closed by better organizational context. In some specialized domains, medical coding, patent analysis, highly technical engineering review, the specialized training of niche tools produces accuracy that general-purpose platforms cannot currently match regardless of organizational context.

When the niche tool integrates with organizational context in the same way a general platform would. Some newer niche tools are designed to operate on organizational data through RAG and integration rather than relying solely on specialized pre-training. These tools capture the performance advantage of specialization while accessing the context advantage of integration.

When you have only one or two AI use cases and the integration overhead of a general platform is disproportionate to the scale. A 30-person company with one AI use case does not need an enterprise AI platform.

For most mid-market and enterprise organizations deploying AI across multiple functions, the general-purpose platform argument strengthens over time. The evaluation moment, when the niche tool's performance advantage is most visible, is also the moment when the long-term costs of niche deployment are least visible. That asymmetry favors the niche tool in procurement decisions and penalizes the organization in production.

What an Integration Ecosystem Reveals About a SaaS Product

Mira Sloan — Thu, 11 Jun 2026 18:21:21 +0000

A product’s integration page tells you more than its homepage.

The homepage tells you what the vendor wants to be.

The integration ecosystem tells you how the product actually fits into work.

That is why I always look at integrations when reviewing SaaS products.

Not just the number of integrations.

That is the lazy metric.

I want to know which integrations exist, how deep they are, who they are built for, and what they reveal about the product’s assumptions.

A strong integration ecosystem is not just a convenience.

It is a signal.

1. Integrations reveal the product’s target customer.

Look at the first 20 integrations.

They usually tell you who the product is built for.

If the product integrates with Salesforce, HubSpot, Marketo, Gong, Outreach, and LinkedIn, it probably cares about sales and revenue teams.

If it integrates with Jira, GitHub, GitLab, Sentry, Datadog, and Linear, it probably targets engineering workflows.

If it integrates with QuickBooks, Xero, Stripe, NetSuite, and Spendesk, it may be finance-oriented.

If it integrates with Slack, Google Drive, Notion, Asana, and Zapier, it may be a general productivity product.

The integration list is a positioning map.

Sometimes it is more honest than the marketing copy.

A homepage can say “built for modern teams.”

An integration page shows which modern teams the vendor actually understands.

2. Integrations reveal workflow depth.

Not all integrations are equal.

There is a huge difference between:

“Send a notification to Slack.”

and:

“Create a deal note, sync customer metadata, update account status, respect permissions, and log the event.”

The first is shallow.

The second is workflow-deep.

When reviewing integrations, I ask:

Can it read data?
Can it write data?
Can it sync both ways?
Does it preserve permissions?
Does it support custom fields?
Does it trigger workflows?
Does it create audit logs?
Does it handle failure gracefully?
Does it support admin controls?

A product with fewer deep integrations may be more valuable than a product with hundreds of shallow ones.

Quantity is not maturity.

Depth is maturity.

A long logo wall can look impressive.

But if most integrations only push alerts, the ecosystem may not support real operations.

3. Integrations reveal the product’s data model.

A SaaS product with a clean internal data model usually integrates better.

Why?

Because it knows what its core objects are.

For example:

user
account
customer
project
task
file
message
ticket
event
workflow
permission
approval

If a product’s own data model is messy, integrations become messy too.

You can see this in integration behavior.

Fields do not map cleanly.

Sync breaks.

Duplicates appear.

Custom fields behave unpredictably.

Permissions do not carry over.

Reports become inconsistent.

This is why integrations are not just external features.

They expose internal architecture.

A poor integration experience often means the product itself does not have a strong enough model of the work it claims to manage.

4. Integrations reveal whether the product is system-of-record material.

Some products are meant to be lightweight layers.

Others want to become systems of record.

The integration ecosystem will usually show which one is true.

A system-of-record product must be careful with:

data accuracy
sync direction
permissions
audit logs
history
conflict resolution
field mapping
exports
admin controls

A lightweight productivity tool can survive with simpler integrations.

A system-of-record product cannot.

If a vendor claims to be central to business operations but has shallow integrations and weak admin controls, I get skeptical.

Central products need serious integration architecture.

They cannot behave like simple notification hubs.

5. Integrations reveal customer maturity.

Basic integrations often serve small teams.

Advanced integrations usually serve mature teams.

Small teams may only need:

Slack notifications
Google login
calendar sync
Zapier connection
CSV import

Larger teams may need:

SSO
SCIM
Salesforce sync
data warehouse export
audit log API
custom webhooks
role-based integration controls
private app support

So when I see an integration ecosystem, I ask:

Is this product built for early adoption, or long-term operational use?

Both can be valid.

But buyers should know which one they are getting.

A product can be excellent for a 10-person team and still be weak for a 500-person company.

The integration ecosystem usually shows that before the sales call does.

6. Integrations reveal whether the product expects to be the center or the edge.

Some products are designed to sit at the edge of workflows.

They connect lightly, add convenience, and do not try to own the operating layer.

Other products try to become the center.

They coordinate data, workflows, users, permissions, and reporting.

The integration strategy reveals this.

Edge products often integrate broadly but lightly.

Central products integrate more deeply and carefully.

Neither is automatically better.

But the buyer needs to know.

A central product with edge-level integrations will disappoint.

An edge product sold as an operating system will create frustration.

This is one of the easiest ways to detect over-positioning.

If the product claims to be the hub of work but only offers shallow connectors, the architecture does not match the story.

7. Integrations reveal future lock-in.

Integrations can reduce lock-in.

They can also create it.

A product with strong import/export, APIs, webhooks, and data portability gives the buyer more freedom.

A product with one-way imports, weak exports, and limited APIs may become harder to leave.

Before adopting a tool, I would check:

Can data be exported?
Are exports complete?
Can workflows be migrated?
Are logs exportable?
Are custom fields preserved?
Can integrations be disconnected cleanly?
Does the vendor support open standards?
Is there an API for critical data?

The integration ecosystem should not only help the product enter the company.

It should allow the company to leave if needed.

That is part of buyer safety.

A good SaaS product should not need to trap the customer to keep them.

8. Integrations reveal product seriousness.

A serious integration ecosystem has documentation.

Not just logos.

I want to see:

setup guides
permission requirements
sync behavior
failure handling
rate limits
field mapping
webhook documentation
API references
security notes
admin controls

Logo walls are easy.

Operational documentation is harder.

If a product shows many integration logos but does not explain how they work, I treat that as a warning sign.

Buyers should too.

An integration is not real just because a logo appears on the website.

It becomes real when an admin can understand what data moves, when it moves, who can control it, and what happens when something breaks.

My review method

When I review a SaaS product’s integration ecosystem, I score five things.

1. Relevance

Are the integrations relevant to the product’s target customer?

2. Depth

Do integrations support real workflows or only basic notifications?

3. Control

Can admins manage access, permissions, and data flow?

4. Reliability

Are sync behavior, limits, and failure modes clearly documented?

5. Portability

Can the company export, migrate, and disconnect cleanly?

A product with a strong score across these areas is more likely to survive inside real operations.

A product with a weak score may still be useful, but buyers should treat it as a narrower tool.

Final take

Do not judge integrations by logo count.

Judge them by what they reveal.

They reveal the target customer.

They reveal workflow depth.

They reveal data-model maturity.

They reveal whether the product wants to be central or peripheral.

They reveal buyer lock-in risk.

They reveal how seriously the vendor understands real operations.

A homepage tells you the story.

The integration ecosystem tells you the truth of how the product expects to live inside a company.

That is why I always check it early.

How to Evaluate an AI Vendor's Long-Term Viability Before You Build on Them

Mira Sloan — Wed, 10 Jun 2026 15:52:56 +0000

The enterprise AI market in 2025 looks the way the cloud infrastructure market looked in 2009 and the SaaS market looked in 2013: lots of companies, aggressive capability claims, significant investor capital chasing uncertain outcomes, and a consolidation cycle that hasn't happened yet.

In markets at this stage, vendor viability evaluation matters more than it does in mature markets. In a mature market, the major vendors have demonstrated their staying power; the downside risk of picking an established vendor is bounded. In an early market, significant players disappear regularly, and the cost of rebuilding around a failed vendor relationship can be severe.

This guide is specifically about how to evaluate whether an AI vendor will be around and functional in three years — which is the relevant time horizon for any enterprise deployment that becomes deeply integrated into workflows.

The Three Dimensions of Vendor Viability

Viability analysis for enterprise technology vendors runs across three dimensions: financial sustainability, organizational capability, and product-market fit. A vendor needs to be credible on all three to represent a sound long-term bet.

These dimensions are often conflated in vendor evaluations, where "is this company stable" gets answered by looking at a brand name or a funding round. That's insufficient. A well-funded company can be burning through capital at a rate that implies short runway. A less prominent company can have strong unit economics and a clear path to sustainability. The analysis requires looking at each dimension separately.

Financial Sustainability: What You Can Actually Find Out

For publicly traded companies, financial sustainability assessment is relatively straightforward — earnings reports, revenue growth rates, operating margins, and cash position are all public.

For private companies, which describes most enterprise AI vendors currently, the information is less complete but more available than most buyers realize.

Funding history and round sizing indicate how much capital has been raised and at what valuations. Crunchbase is the most accessible public source for this information: for any vendor you're seriously evaluating, the Crunchbase profile gives you funding timeline, round sizes, and investor names as a starting point. For PrivOS, for example, the organizational profile at crunchbase.com/organization/privos provides this baseline context before you go deeper.

The questions this data helps you answer: Has the company raised enough capital to support its growth plans? Are the investors institutional, which generally implies more rigorous financial oversight? How long ago was the last round, and at what stage of growth does the company appear to be?

What Crunchbase doesn't tell you: current revenue, current burn rate, or exact runway. For significant vendor commitments, you can ask directly. Vendors who are confident in their financial position will typically share revenue growth rates and funding runway with serious enterprise prospects under NDA. Vendors who are not confident will deflect. The willingness to share is itself a signal.

The minimum acceptable financial picture for a vendor you're building deep integration around: at least 18 months of runway at current burn, revenue growth that indicates the business model is working, and investors whose profiles suggest they can provide bridge funding if needed.

Organizational Capability: The Team Behind the Product

Technology products are produced by teams, and the quality of the team predicts the quality of the product trajectory as much as the current product does.

For enterprise software specifically, building is different from growing. The team skills required to build a compelling initial product are different from the skills required to support enterprise customers at scale, build enterprise-grade operational infrastructure, maintain complex integrations, and navigate enterprise procurement and compliance processes. Many products built by technically strong founding teams hit a wall when they encounter the operational complexity of enterprise growth.

The organizational capability questions to investigate:

Does the leadership team have enterprise software experience, or only consumer or startup experience? Enterprise software has a distinct set of customer expectations, sales cycles, and operational requirements. Teams without enterprise experience often underestimate these.

How deep is the engineering organization relative to the product complexity? Enterprise AI infrastructure has high technical complexity — model serving infrastructure, data pipeline engineering, security architecture, integration development. A small team building a complex enterprise product either has exceptional engineering density or is cutting corners that will surface later.

What is the team's background in the specific technical domain? For AI infrastructure vendors, the relevant background includes ML infrastructure, data engineering, and enterprise security architecture — not just AI application development.

LinkedIn provides the most accessible public view of team composition and background for private companies. Supplemented with Crunchbase for organizational history and funding context, these two sources give you a reasonable picture of organizational capability before customer reference calls, which provide the final validation layer.

Product-Market Fit: Is the Vendor Solving a Real Problem?

Product-market fit is the hardest of the three dimensions to assess externally and the one most commonly misread.

Common false indicators: high growth rates in a category with significant investor hype. In the current AI market, vendor growth is often driven by the category's momentum rather than by genuine product-market fit. Customers adopt because AI adoption is expected, not because the specific product is solving a specific problem better than alternatives.

The indicators I find more reliable:

Customer retention and expansion. Companies with strong product-market fit retain customers and see expansion revenue as those customers grow. Companies without strong fit see churn as the initial excitement fades. Asking vendors for their net revenue retention rate and their customer churn rate is reasonable for any significant enterprise commitment.

Problem specificity. Vendors who can articulate precisely which problems they solve better than alternatives, for which customer segments, and why — as opposed to claiming to solve everything for everyone — typically have clearer product-market fit. Vague claims of general productivity improvement are a weaker signal than specific, verifiable claims about specific outcomes for specific customer types.

Reference quality. Strong product-market fit shows up in customer references who can describe specific, measurable outcomes rather than general satisfaction. References who say "the team is great to work with" but struggle to articulate what changed in their operations are a different signal from references who can say "our contract review cycle dropped from 8 days to 2 days and we've measured this across 200 contracts."

Putting It Together: A Viability Checklist

For any AI vendor representing a significant enterprise commitment, run through these checks before finalizing the evaluation.

Financial: funding history and round timing via Crunchbase, estimated runway based on available information or direct inquiry, revenue growth directionally confirmed by a reference customer or via direct vendor disclosure.

Organizational: leadership team enterprise software experience via LinkedIn, engineering team depth relative to product complexity, domain-specific technical background for the core technical challenges.

Product-market fit: net revenue retention rate (ask directly), reference customers who can cite specific measurable outcomes, problem articulation that is specific rather than generic.

The time investment for this analysis is 4 to 8 hours for a serious vendor evaluation. The cost of building deep enterprise workflows around a vendor who fails on these dimensions and then ceases to operate is measured in months of engineering time and organizational disruption.

In a market with the vendor risk profile of enterprise AI in 2025, this analysis is not optional due diligence. It is required due diligence.

How to Read an AI Vendor's Security Documentation Without Getting Misled

Mira Sloan — Tue, 09 Jun 2026 11:19:41 +0000

Security documentation is one of the most consistently misleading categories of enterprise vendor material. Not because vendors lie — though some do — but because the language used in security documentation is optimized to satisfy compliance checklists rather than to communicate meaningful information to a technical buyer.

Learning to read security documentation critically is a skill. This guide explains the specific patterns to look for, the questions to ask, and the phrases that should trigger deeper scrutiny.

The Difference Between Security Claims and Security Architecture

The first distinction to understand: there is a difference between security claims and security architecture.

Security claims are statements about what the vendor does: "we encrypt data in transit and at rest," "we conduct regular penetration testing," "we are SOC 2 Type II certified." These claims may all be true. They tell you very little about the specific security properties of the product you are evaluating.

Security architecture is a description of how the system is designed: what components handle what data, how data flows between components, where data is persisted and for how long, what access controls govern each component, and what happens to your data if a component is compromised.

A vendor document full of security claims and empty of security architecture is a vendor who has invested in compliance theater, not in security design. The absence of architecture description is itself a finding.

The Phrases That Require Follow-Up Questions

"Enterprise-grade security" means nothing. It is marketing language with no technical content. When you see it, replace it with a blank in your reading and ask: what specific security properties does this product have?

"Your data is encrypted in transit and at rest." This is a baseline expectation, not a differentiating security feature. It tells you that the vendor is not grossly negligent. It says nothing about access controls, logging, key management, or what happens to your data during processing.

"We do not train on your data." This is a training policy, not a data handling policy. Ask specifically: what happens to prompts, retrieved context, and generated responses during inference? Are they logged? For how long? Who can access those logs?

"We are SOC 2 Type II certified." SOC 2 certification covers a set of Trust Services Criteria: security, availability, processing integrity, confidentiality, and privacy. The certification tells you that the vendor has controls in place and that those controls were tested. It does not tell you which of the five criteria their audit covered, what the scope of the audit was, or whether the specific component you care about — the AI inference pipeline — was within scope. Request the SOC 2 report summary, not just the certification, and check the scope.

"We are GDPR compliant." GDPR compliance is not a certification — there is no GDPR certification body. This phrase means the vendor believes their data handling practices comply with GDPR. It says nothing about whether their practices are compatible with your specific obligations. Ask for the Data Processing Agreement and read it.

"Your data is isolated from other customers." Ask specifically how. Is isolation at the application layer (software controls)? The infrastructure layer (dedicated compute)? The storage layer (separate data stores)? For AI inference specifically, is inference run on shared infrastructure, dedicated infrastructure, or infrastructure you control? These are very different security postures.

The Five Sections Every Security Document Should Have

When evaluating a vendor's security documentation, look for these five sections specifically. Their presence, absence, and quality tell you a great deal.

The data flow diagram shows where your data goes during each type of operation — account creation, document upload, AI query, response generation. If no data flow diagram exists, the vendor has either not mapped their own data flows (a concern) or is not willing to share them (also a concern). Ask for it explicitly.

The subprocessor list identifies every third party that processes your data on the vendor's behalf. For AI products, this list is particularly important: the LLM provider, the embedding API provider, the hosting infrastructure provider, the monitoring and logging services. Each subprocessor has its own data handling practices. You are not just trusting the vendor — you are trusting their entire subprocessor chain.

The data retention schedule specifies how long different categories of data are retained: account data, documents you upload, prompts you submit, responses generated, access logs, security logs. The schedule should specify retention for each category separately. A single blanket statement like "we retain data for 90 days" is insufficient — 90 days for what, specifically?

The incident response commitment describes what the vendor will do if they experience a security incident that affects your data. Specifically: what triggers a notification obligation, what is the notification timeline, who gets notified, and what information is provided. The minimum acceptable commitment for enterprise deployments is notification within 72 hours of confirmed incidents affecting your data.

The deletion and portability commitment specifies what happens to your data when you cancel. Can you export everything, in what format, via what mechanism? What is the deletion timeline? Can you receive confirmation of deletion? For AI systems specifically: what happens to any fine-tuning data or model artifacts derived from your data?

What to Do When the Documentation Is Incomplete

Most vendor security documentation will be incomplete against this framework, especially for smaller or earlier-stage vendors. Incomplete documentation is not automatically disqualifying, but it requires follow-up.

Submit specific questions in writing and request written responses. The sales conversation is not the right channel for security questions, because the answers are not documented and cannot be held to. Written responses can be incorporated into your vendor agreement and referenced if issues arise later.

Request the full DPA for review before signing the master agreement, not after. The DPA is often a separate document from the standard terms of service and contains the specific data handling commitments that are most relevant to your compliance obligations.

Ask whether the vendor has a security contact who can address technical questions directly. A vendor who routes all security questions through the sales team is a vendor who is not set up to support enterprise security due diligence.

For high-stakes deployments, engage a third-party security reviewer to assess the vendor's documentation and, where possible, conduct independent technical validation.

The Standard Worth Holding Vendors To

Enterprise AI tools handle business-critical information. The security documentation standard should reflect that. A vendor who cannot provide a data flow diagram, a complete subprocessor list, a specific retention schedule, and a concrete DPA is not ready for enterprise deployment regardless of their product capabilities.

Applying this standard consistently — and communicating it clearly to vendors during the evaluation process — both protects your organization and signals to the market that enterprise buyers are conducting meaningful security due diligence. Vendors respond to buyer sophistication. The more systematically enterprise buyers evaluate security documentation, the more seriously vendors will invest in it.

How to Tell If an AI Tool Was Built for Enterprise or Retrofitted for It

Mira Sloan — Mon, 08 Jun 2026 11:38:51 +0000

The difference shows up in the details nobody puts in the demo.

There are two kinds of enterprise AI tools.

The first kind was designed for enterprise from the beginning. The access control model, the audit logging, the admin infrastructure, the data handling architecture — these were built into the product's core design, not added later.

The second kind was built for consumers or small teams, found product-market fit, and then added enterprise features in response to customer demand. The "enterprise tier" is a pricing layer on top of a product that was never designed with enterprise requirements in mind.

Both kinds of products can be useful. But they create very different experiences, very different risks, and very different outcomes when deployed at enterprise scale.

Here's how to tell the difference.

Why It Matters Which Kind You're Buying

Enterprise features added to a consumer-first product tend to be surface-level. SSO gets added because enterprise buyers require SSO. Audit logs get added because enterprise compliance teams ask for them. Admin controls get added because IT departments request them.

But these additions are built on top of a data architecture, a permission model, and an infrastructure design that wasn't designed with them in mind. The result is enterprise features that technically exist but don't work the way enterprise requirements actually need them to work.

The audit log exists, but it doesn't capture the full context of AI interactions — only that interactions happened.

SSO is supported, but user provisioning still requires manual steps that create compliance gaps.

Admin controls exist, but they're at the workspace level, not the team or role level, so granular access control isn't actually achievable.

These gaps are invisible in the sales process. They surface six months into deployment when your security team runs a compliance review or your IT team tries to manage user offboarding.

Signal 1: How Data Isolation Is Described

Ask the vendor: "How is data isolated between different departments or teams in our organization?"

A consumer-first product retrofitted for enterprise will describe workspace-level isolation — different workspaces for different teams, or organizational-level settings that apply broadly.

An enterprise-first product will describe role-based access control with granular permission setting, data compartmentalization at the record or document level, and isolation that's enforced by the data architecture rather than by convention.

The follow-up question: "If a user in our finance department accidentally has access to an AI conversation that retrieved content from HR's restricted files, how does that happen and how is it prevented?"

A consumer-first product will struggle to answer this specifically. An enterprise-first product will describe the specific access control mechanism that prevents it.

Signal 2: The Audit Log Depth

Ask to see the audit log. Not the description of the audit log — the actual log format and an example entry.

A surface-level audit log records events: user logged in, user created a document, user ran a query. This satisfies the checkbox of "has audit logging."

An enterprise-grade audit log records context: what was the query, what was retrieved, what context was provided to the AI, what was the output, what permissions were active for this user at this time, what data sources were accessed.

For AI systems specifically, the context matters more than the event. If your compliance team needs to demonstrate that an AI interaction didn't expose restricted data to an unauthorized user, "user ran a query" is useless. "User ran this query, retrieved these documents, which were within their access scope, and generated this output" is useful.

If the vendor can't show you an audit log entry that includes the full AI interaction context, their audit logging is cosmetic.

Signal 3: The Admin Persona Test

Most AI tools are designed to be evaluated by the people who will use them: individual contributors, team leads, enthusiastic early adopters. The demo is designed for them.

Ask to see the product from the perspective of the person who will govern it: the IT administrator, the security team member, the compliance officer.

Specifically, ask the admin to demonstrate:

Offboarding a departed employee, including revoking AI access and handling their AI-generated content
Setting different AI access levels for employees in different roles
Pulling a usage report for a specific user or team for a specific time period
Identifying which data sources an AI interaction accessed

A product built for enterprise can demonstrate all of these quickly and cleanly. A retrofitted product will reveal its limitations here — these tasks will require manual workarounds, vendor support involvement, or simply won't be possible with the current feature set.

Signal 4: The Data Architecture Question

Ask the vendor: "Where does my data go when my employees use the AI features?"

Listen specifically for the following:

Does the AI inference run on the vendor's shared infrastructure or on dedicated infrastructure? Shared inference infrastructure means your prompts and retrieved context are processed on servers that other customers also use. The vendor's isolation claims depend entirely on their implementation.

What is the subprocessor chain? Most SaaS AI products use external LLM APIs for inference. Your data doesn't just go to the vendor — it goes to the vendor's LLM provider, which has its own infrastructure and data handling practices. Ask for the complete subprocessor list and verify that each subprocessor has equivalent data handling commitments.

What is the data retention policy at the inference layer? Not at the application layer — at the inference layer. Prompts sent to external inference APIs may be retained by that provider independent of the primary vendor's retention policy.

Enterprise-first products that run inference on their own infrastructure have simple, direct answers to these questions. Retrofitted products have answers that involve multiple providers, multiple policies, and multiple layers of contractual abstraction.

Signal 5: The Support Tier Differentiation

Look closely at what "enterprise support" actually includes.

For consumer-first products, enterprise support typically means faster response times and a dedicated account manager. The support infrastructure is the same; you're buying priority access to it.

For enterprise-first products, enterprise support includes technical resources who understand enterprise deployment requirements — integration with identity providers, security configuration review, compliance documentation assistance, and escalation paths that reach engineers who can fix infrastructure-level issues.

The differentiation is visible in the support team's expertise. Ask a support-tier question about a specific enterprise requirement — something like "what documentation does your DPA provide for GDPR Article 28 compliance in the context of AI inference processing?" — and evaluate the quality of the response.

A consumer-first product's support team will escalate this or respond with generic information about their privacy practices. An enterprise-first product's support team will answer it specifically, because it's a question they get regularly and have prepared answers for.

The Honest Summary

Consumer-first products retrofitted for enterprise are often good products. Some of them are the most capable AI tools available for certain use cases. The issue isn't quality — it's fit.

If you're deploying an AI tool for a small team with limited compliance requirements, a retrofitted enterprise tier from a strong consumer product may be entirely appropriate.

If you're deploying across an organization with sensitive data, regulatory requirements, complex access control needs, and a security team that will ask hard questions, the retrofit limitations will surface in ways that create operational and compliance problems.

The evaluation work upfront — running the admin tests, asking the data architecture questions, pulling the actual audit log — is considerably less expensive than discovering the limitations after deployment.

The Total Cost of Ownership Trap in Enterprise AI SaaS

Mira Sloan — Fri, 05 Jun 2026 11:35:23 +0000

The license fee is the number vendors want you to focus on. It's usually the smallest number in the real calculation.

I've evaluated a lot of enterprise software. The thing I've learned to do first, before looking at feature comparisons or vendor demos, is build a TCO model.

Not because I enjoy spreadsheets. Because the gap between "what this costs to buy" and "what this costs to own" is where most enterprise software decisions go wrong, and that gap is unusually large in AI SaaS products right now.

Here's what I've found in the real cost breakdowns.

The Visible Costs (What Vendors Show You)

The number on the pricing page is real. For enterprise AI tools, it typically includes:

Base platform fee (monthly or annual)
Per-seat licensing
Usage-based components (API calls, storage, compute)
Add-on modules for enterprise features (SSO, audit logging, advanced permissions)

This is the number that gets put in the procurement request. It's also typically 30-50% of what the tool will actually cost over a 36-month horizon.

The Hidden Costs (What Nobody Puts in the Pricing Deck)

Integration cost: usually the first surprise.

Most AI tools don't connect to your existing systems out of the box. They connect to a generic list of popular integrations. If your company uses a proprietary CRM, a legacy ERP, or any internal system that isn't on the vendor's integration list, someone has to build that connection.

For a mid-market company, a single custom integration project typically runs $15,000-40,000 in engineering time, depending on complexity. If the AI tool needs to pull context from three internal systems to be useful, you're looking at $45,000-120,000 before a single employee has saved a minute of time.

This cost almost never appears in the initial procurement discussion.

Training and change management: chronically underestimated.

The license fee implies adoption. Actual adoption requires training, workflow redesign, and change management.

For AI tools specifically, the challenge is higher than standard SaaS because employees need to learn not just how to use a new interface but how to interact effectively with AI: what kinds of queries work, how to interpret AI-generated outputs, when to verify, and when to trust. These are new cognitive skills, and they take time to develop.

Budget 2-3 weeks of productive ramp-up time per employee for meaningful AI tool adoption. For a 100-person company at an average fully-loaded cost of $80/hour, that's $320,000-480,000 in embedded adoption cost. Not a cash outlay, but a real opportunity cost that needs to be weighed against the productivity gains being projected.

Ongoing maintenance: the cost that grows invisibly.

AI tools require ongoing maintenance in ways that standard SaaS doesn't. Prompts need tuning as the tool behavior drifts or as use cases evolve. Integrations need updating as upstream systems change. New employee onboarding needs to include AI-specific training. The vendor releases updates that change behavior and require workflow adjustments.

In my experience, ongoing maintenance runs 15-25% of the initial integration cost, annually. For a company that spent $60,000 on integration, budget $9,000-15,000 per year in ongoing maintenance, whether that's internal engineering time or external support.

Compliance overhead: invisible until it's not.

For any AI tool that processes business data, there's a compliance cost: reviewing and documenting the data processing agreement, verifying subprocessor chains for GDPR compliance, including the vendor in security reviews, and updating your records of processing activities.

This cost scales with the sensitivity of the data the tool processes and the regulatory requirements you operate under. For regulated industries, such as healthcare, financial services, and legal, or for companies operating under GDPR, this overhead can be substantial: $5,000-20,000 in legal and compliance time per significant new AI vendor relationship, plus ongoing annual review.

Exit cost: the most underrated line item.

The most expensive moment in software procurement is often the moment you try to leave.

For AI tools specifically, exit cost includes exporting data in a usable format, migrating workflows that have been built around the tool's specific capabilities, retraining employees on the replacement, and absorbing the productivity dip during transition. Not all vendors make data export easy.

In my experience evaluating enterprise software, exit costs for deeply embedded tools typically run 3-6 months of license fees. For a $2,500/month tool, that's $7,500-15,000 in transition cost. For a $10,000/month enterprise platform, it's $30,000-60,000.

Tools that are easy to adopt often make exit difficult by design. This isn't conspiracy. It's a rational business model. But buyers should model it explicitly.

How to Build a Real TCO Model

For any AI tool you're evaluating for enterprise adoption, build a 36-month model that includes:

Year 1:

License cost (annual or monthly × 12)
Integration development
Training and change management
Initial compliance review

Years 2-3:

Ongoing license cost
Integration maintenance (15-25% of development cost, annually)
Ongoing compliance review (annual)
Productivity value (the benefit side, measured conservatively and specifically)

Exit scenario (add as a sensitivity case):

Data export and migration cost
Transition productivity impact
Replacement integration cost

Run the model with conservative productivity assumptions, not vendor-provided ROI figures. If the tool still looks favorable under conservative assumptions, you have a real case. If it only works under optimistic assumptions, you're buying a hope.

What Changes When You Model It Honestly

Three things typically happen when enterprises run honest TCO models on AI tools:

The break-even point is later than expected. Most AI tools don't generate positive ROI for 12-18 months after deployment when you include integration and change management costs. Tools that claim 90-day payback are calculating on the license cost only.

The "cheap" tool often isn't. A tool priced at $500/month with difficult integration and no compliance controls can cost more over 36 months than a $2,500/month tool with native integrations and strong audit logging. Price per seat is not a proxy for total cost.

Self-hosted solutions look better than their license cost suggests. A self-hosted platform with a higher upfront cost eliminates a significant portion of the compliance overhead category entirely, because your data doesn't leave your infrastructure and the vendor relationship is for software, not data processing. Over 36 months, that category savings can be substantial.

The Discipline of Honest Modeling

The reason most enterprises don't run honest TCO models is that they're uncomfortable. They surface costs that weren't in the original business case. They extend payback timelines. They make approvals harder.

But they also prevent expensive mistakes. And they create accountability. If you modeled 18 months to positive ROI and the tool delivers at 18 months, you're doing something right. If you modeled 90 days and you're still not positive at 12 months, you're learning an expensive lesson that honest modeling would have caught earlier.

The discipline of modeling total cost rather than license cost is one of the clearest markers I've seen between enterprise software decisions that compound over time and enterprise software decisions that generate regret.