DEV Community: James Glaser

Building an AI That Knows When Not to Guess

James Glaser — Fri, 23 Oct 2026 20:46:15 +0000

Building an AI That Knows When Not to Guess

Most AI chatbots have a pretty simple job: answer the question.

Ask about programming, productivity, or a new technology, and a chatbot can usually give you something useful to work with.

But what happens when the question is about the law?

Someone might be trying to understand what happens after a car accident, whether a landlord can keep a security deposit, or what options they have after getting injured at work.

A confident but inaccurate answer isn't just inconvenient. It could influence a decision that has real consequences.

That's why building AI for legal information presents a different kind of engineering challenge.

The goal isn't just to generate an answer. It's to generate an answer that deserves to be trusted.

The Problem With AI That Always Has an Answer

One of the interesting things about large language models is how naturally they can explain complicated subjects.

They can turn technical language into plain English, summarize long documents, and respond to follow-up questions without requiring users to learn specialized terminology.

But fluency and accuracy aren't the same thing.

An AI system can produce a convincing explanation while getting an important detail wrong. It might overlook an exception, confuse two jurisdictions, or cite a legal authority that doesn't support its conclusion.

In everyday conversations, these mistakes might be easy to correct.

In legal information, the consequences can be much more serious.

Imagine asking an AI assistant whether you have time to file a personal injury claim. A generic answer about typical deadlines might sound reasonable, but the applicable rules could depend on the type of claim, the circumstances, and other legally significant details.

A useful system needs to recognize those limitations rather than hide them behind confident language.

Context Matters More Than We Think

Consider two people asking the same question:

"Can I sue after a car accident?"

The first person was involved in a collision in Massachusetts yesterday.

The second was involved in an accident in another state several years ago.

The words are identical, but the relevant legal framework and next steps may be very different.

This illustrates a broader lesson in AI development: an answer can be factually plausible and still be wrong for the person asking the question.

A well-designed system needs to establish the boundaries of its knowledge.

For a legal AI assistant, that means paying attention to jurisdiction, identifying the relevant legal topic, and distinguishing general information from guidance that requires an attorney to evaluate specific facts.

Adding more conversational ability won't solve a missing-context problem by itself.

Sometimes the most important improvement is making sure the system understands which question it is actually answering.

Why We Can't Treat Every Chatbot the Same Way

A general-purpose chatbot and a specialized legal information system have different responsibilities.

A general assistant might draw on broad knowledge to explain a concept. A legal information system needs a more disciplined approach to its sources, scope, and limitations.

For example, a specialized system can be designed around a curated collection of statutes and case law relevant to a particular jurisdiction.

Instead of relying entirely on whatever the model recalls from training, the application can retrieve relevant legal authorities and use them to construct an answer.

This approach doesn't eliminate every risk. The system can still select an irrelevant source, misinterpret a provision, or overlook a fact that changes the analysis.

But it creates a more structured process for grounding the response and checking whether its claims are supported.

It also makes it easier to ask an important question:

Where did this answer come from?

What Happens When AI Has Boundaries?

A lot of AI development focuses on expanding capabilities.

More tools. More integrations. Longer context windows. More sophisticated reasoning.

Those improvements can be valuable, but specialized applications also benefit from deliberate restrictions.

A legal information system might need to:

Stay within the laws of a defined jurisdiction.
Use identifiable, relevant legal authorities.
Avoid inventing citations.
Explain when the available information is insufficient.
Distinguish legal information from individualized legal advice.
Direct people to a licensed attorney when their circumstances require professional evaluation.

These aren't merely limitations on what the AI can do.

They're part of how the product is designed to work responsibly.

For certain applications, a system that knows when to stop is more useful than one that attempts to answer everything.

A Practical Example: Making Legal Information Easier to Access

This is the problem that projects like Jimmy Knows AI are exploring.

The platform provides a public-facing resource for Massachusetts legal information, combining explanatory content with an AI feature that answers questions using a curated body of relevant legal authorities.

The idea is straightforward: people should have a way to begin understanding a legal issue without first having to interpret statutes and court decisions on their own.

Someone who has been injured in an accident, for example, may not know which questions to ask or which legal concepts are relevant.

A plain-language explanation can give that person a starting point and help them understand what information might matter next.

But the distinction is important: the platform provides legal information, not a substitute for an attorney evaluating the facts of an individual case.

The AI can help explain the landscape. A licensed attorney can evaluate the circumstances that determine what options may actually be available.

That separation between information and professional judgment is an important part of the design.

The Engineering Doesn't Stop at the Model

There's another lesson here for developers building specialized AI applications.

The model is only one component of the system.

A production-ready application also needs to consider:

Source management. Legal references need to be relevant, traceable, and maintained as laws and decisions change.

Retrieval. The system needs to find the appropriate authorities for the question rather than simply return the first vaguely related passage.

Validation. Responses can be checked for issues such as unsupported citations, missing disclaimers, and claims outside the system's intended scope.

User experience. People need to understand what the system can help with, what it cannot determine, and when they should seek professional assistance.

Maintenance. A legal information product cannot assume that the information it was built with will remain correct forever.

These concerns apply well beyond legal technology. Medical information systems, financial applications, compliance tools, and other high-stakes AI products face similar challenges.

The more consequential the decision, the less acceptable it becomes to rely on fluent output alone.

Trust Is a Product Feature

It's tempting to think of trust as something a company earns through branding, reputation, or a polished user interface.

In AI applications, trust also depends on the system's behavior.

Does it distinguish evidence from assumptions?

Does it explain uncertainty?

Can users identify the sources behind its claims?

Does it stay within its intended scope?

Does it admit when it cannot reliably answer a question?

These details shape the experience just as much as response speed or conversational quality.

A chatbot that responds instantly but gives users no way to evaluate its claims may feel impressive during a demo. That doesn't necessarily make it dependable.

In contrast, a system that provides a clear answer, identifies its supporting authorities, and acknowledges its limitations gives users more information with which to judge the response.

That is a much stronger foundation for trust.

The Bigger Lesson for AI Developers

The next wave of useful AI products may not come from making general-purpose models sound even more human.

It may come from designing better systems around them.

Systems with better data, clearer boundaries, stronger evaluation, and a more thoughtful understanding of the decisions users are trying to make.

A model can generate an answer.

Building a useful product means deciding which answers it should generate, what evidence should support them, and what should happen when the evidence isn't enough.

That requires engineering, domain knowledge, and an honest understanding of the technology's limitations.

Final Thoughts

AI has enormous potential to make complicated information easier to access.

But accessibility alone isn't enough when people may use that information to make important decisions.

The system also needs to be careful about what it says, where its answers come from, and when a human professional needs to take over.

That's what makes specialized AI interesting to build.

The challenge isn't simply teaching a model to answer questions about a particular subject.

It's designing an application that understands the boundaries of its role.

The most useful AI isn't necessarily the one that answers every question. It's the one that helps people understand what they can know, what they still need to verify, and what they should do next.

Your Legal Question Has an Address

James Glaser — Tue, 18 Aug 2026 20:59:22 +0000

Most people don't think about location when they ask a legal question.

They think about the event.

“I was hit by a car.”

“My landlord kept my deposit.”

“I slipped on ice.”

“I got hurt at work.”

But there is another question hiding inside all of these:

Where did it happen?

That question can completely change the answer.

Law isn't just about what happened

If someone asks a general AI:

“Can I sue after slipping on ice?”

The model might produce a reasonable-sounding explanation about premises liability.

But reasonable isn't necessarily useful.

The rules governing that situation depend on the jurisdiction.

Even something as specific as Massachusetts snow and ice law has its own history and rules. Massachusetts moved away from the old “natural accumulation” distinction after Papadopoulos v. Target, which changed how these cases are analyzed.

So “I slipped on ice” isn't really a complete legal question.

It is closer to:

“I slipped on ice, in this place, under these circumstances, on this date. What law applies?”

That is a very different problem.

Location should be part of the input

This is one of the things I find interesting about building legal AI.

In a normal chatbot, location might be treated as optional context.

In a legal system, it can be one of the most important pieces of information.

A useful intake process might look something like:

What happened?
↓
Where did it happen?
↓
Which jurisdiction governs?
↓
What authority applies?
↓
What facts are still missing?
↓
Explain the relevant rule

The location isn't simply added at the end.

It helps determine which legal sources should even be considered in the first place.

Massachusetts is a good example

Jimmy Knows is intentionally focused on Massachusetts rather than trying to answer legal questions for every state.

That constraint is actually a feature.

The platform's answers are grounded in Massachusetts statutes and case law, and its content is localized across Massachusetts cities and practice areas.

That means a question about a workplace injury in Brockton isn't treated as an abstract “workers' compensation” question.

The system can start from the fact that the matter is in Massachusetts and work from the relevant Massachusetts authority.

That sounds obvious.

It isn't.

A general-purpose AI has to deal with an enormous amount of potentially conflicting law. A user can ask a perfectly clear question and still get an answer that is based on the wrong jurisdiction.

The answer may even sound convincing.

That's the dangerous part.

The problem isn't always hallucination

When people talk about AI mistakes in legal applications, the conversation usually goes straight to hallucinated cases or invented citations.

Those are serious problems.

But there's another failure mode that is easier to miss:

The AI gives you a real rule from the wrong place.

The statute exists.

The case exists.

The explanation sounds professional.

It is still the wrong answer.

That's why jurisdiction isn't just a filtering feature for legal AI.

It should be part of the reasoning process.

Maybe the best legal AI is narrower

There is a natural temptation to make AI systems broader.

More states.

More practice areas.

The Most Important Feature of a Legal AI Might Be Knowing When Not to Answer

James Glaser — Mon, 17 Aug 2026 20:20:14 +0000

One of the things that gets overlooked when building AI for legal use is uncertainty.

Most AI products are designed to answer questions.

Legal questions are different.

A person might type:

“I got hurt at work. Can I sue my employer?”

That sounds simple, but it isn't.

What happened?
Where did it happen?
Was the person an employee or contractor?
Was another company involved?
Was the injury caused by equipment?
When did it happen?
Has a workers' compensation claim already been filed?

A few missing facts can completely change the legal analysis.

This is why I think legal AI should not be judged only by how many questions it can answer.

It should also be judged by how well it handles questions that it cannot safely answer yet.

The temptation to give a confident answer

Large language models are very good at producing something that sounds complete.

That is useful in many applications.

It can also be a problem in legal applications.

If the system does not have enough information, generating a polished paragraph can create a false sense of certainty.

The better approach is sometimes to stop and ask another question.

Not:

“Here is what you should do.”

But:

“Before I can give you a useful answer, I need to know whether another party was involved.”

That small difference matters.

Building around the facts, not the prompt

For Jimmy Knows, the idea is to start with what actually happened.

A user can describe their situation in their own words instead of trying to figure out which legal category their problem belongs to.

The system can then connect the facts to Massachusetts statutes and case law and explain the relevant rule in plain language.

That creates a different design philosophy:

Don't make the user speak like a lawyer. Make the system understand the user's story.

This also means the AI needs to recognize when the story is incomplete.

For example:

“I slipped outside a store and got hurt.”

There isn't enough information yet.

Where did it happen?

Was there snow or ice?

Was there a visible hazard?

Did the store know about it?

Those questions are not just conversational filler. They can determine which legal rules are relevant.

The interesting engineering problem

The hard part isn't necessarily generating the answer.

It is deciding whether an answer should be generated at all.

That means a legal AI system needs something beyond retrieval and generation.

It needs a concept of sufficient facts.

You can think of the workflow like this:

User tells their story
        ↓
Extract relevant facts
        ↓
Identify missing facts
        ↓
Are the facts sufficient?
     ↙        ↘
   No          Yes
   ↓            ↓
Ask questions   Retrieve authority
                ↓
             Explain rule
                ↓
          Explain next step

This is a much more interesting problem than simply connecting an LLM to a legal database.

The quality of the system depends heavily on what happens before the final answer is written.

AI should reduce uncertainty, not hide it

There is a tendency to think a good AI assistant should always sound confident.

I think the opposite is often true for legal applications.

A trustworthy system should be comfortable saying:

“I don't have enough information yet.”

That doesn't make the system less useful.

It makes the next question more useful.

And eventually, that can lead to a much better answer.

That's one of the ideas behind Jimmy Knows: make Massachusetts legal information easier to access while keeping the distinction between legal information and advice clear. The platform provides information based on Massachusetts authority and connects people with an attorney when they need one.

The future of legal AI probably isn't about making AI sound more like a lawyer.

It may be about making AI better at knowing when the facts are not there yet.

And honestly, that might be one of the hardest parts to get right.

The Best Time to Use AI for Legal Questions Is Before You Have a Legal Problem

James Glaser — Tue, 04 Aug 2026 13:50:33 +0000

Most people turn to legal information when something has already gone wrong.

A contract has been breached.

A landlord sends a notice.

An employee resigns unexpectedly.

A customer threatens legal action.

By that point, every decision feels urgent.

But one of the most valuable uses of AI isn't responding to legal problems. It's helping people avoid them in the first place.

Prevention Is Easier Than Recovery

Businesses invest in cybersecurity before they get hacked.

They buy insurance before accidents happen.

They back up their data before a hard drive fails.

Legal preparation should work the same way.

Understanding legal documents, obligations, and risks before signing an agreement is almost always easier than dealing with a dispute later.

Most Legal Problems Start Small

Rarely does a legal issue appear out of nowhere.

More often, it starts with something that seemed insignificant.

A vague sentence in a contract.

An email that wasn't documented.

An agreement made over the phone.

An employee handbook that was never updated.

A policy copied from another company.

These small details can grow into expensive problems months or years later.

AI Makes Legal Information More Accessible

Legal language can be intimidating.

Many contracts are written for precision, not readability.

That makes it difficult for people without a legal background to understand what they're agreeing to.

AI can help by:

Explaining unfamiliar legal terms.
Summarizing lengthy documents.
Highlighting important clauses.
Breaking complex language into plain English.
Helping users identify questions worth asking.

The goal isn't to replace legal professionals.

The goal is to make legal information less intimidating.

Better Understanding Leads to Better Conversations

Imagine walking into a meeting with your attorney already understanding:

The purpose of the contract.
The key obligations.
The deadlines involved.
The clauses that concern you most.

Instead of spending the first half of the meeting explaining the basics, you can focus on the decisions that actually matter.

That's a better use of everyone's time.

AI Helps You Spot Questions, Not Just Answers

One of the most underrated benefits of AI is that it encourages curiosity.

After reviewing a document, you might ask:

Why is this clause included?
What happens if this deadline is missed?
Can this agreement be terminated early?
Is this language common?
Should this section be negotiated?

Those questions often lead to better outcomes than simply searching for a quick answer online.

The Human Element Still Matters

No AI system understands your complete situation.

It doesn't know your goals, your business relationships, your financial priorities, or every fact surrounding your case.

That's why legal decisions still require human judgment.

AI is best used as a tool for understanding.

Lawyers remain essential for advice, strategy, and representation.

The two work best together.

The Future of Legal Technology

The biggest impact of AI in law may not be replacing legal work.

It may be giving more people the confidence to engage with legal information instead of avoiding it.

When contracts become easier to read and legal concepts become easier to understand, people make more informed decisions.

That benefits individuals, businesses, and legal professionals alike.

Final Thoughts

You don't have to wait until you're facing a legal dispute to benefit from AI.

Sometimes the smartest time to ask legal questions is before you sign, before you agree, and before the problem exists.

Understanding your rights and responsibilities early can prevent misunderstandings, reduce risk, and lead to better decisions long before legal action ever becomes necessary.

Why Every Business Should Read Contracts Like a Risk Manager, Not a Lawyer

James Glaser — Tue, 28 Jul 2026 16:48:45 +0000

Most people read contracts looking for what they expect to see.

Experienced professionals read contracts looking for what could go wrong.

That difference can save thousands of dollars, prevent disputes, and protect long-term business relationships.

You don't need to be a lawyer to develop this mindset. You simply need to start asking different questions.

A Contract Is a Risk Allocation Document

Many people think a contract is simply an agreement.

In reality, it's a document that decides who bears the risk when something doesn't go according to plan.

Every important clause answers questions like:

Who pays if something fails?
What happens if a deadline is missed?
Can either party walk away?
Who owns the work after it's completed?
How will disagreements be resolved?

The answers often matter more than the price of the deal.

The Clauses That Matter Most

Business owners tend to focus on deliverables and payment.

Those are important, but they're rarely the clauses that create expensive disputes.

Pay close attention to:

Termination

Can either party end the agreement?

How much notice is required?

Are there penalties?

Liability

If something goes wrong, who is responsible?

Is liability limited?

Are there financial caps?

Intellectual Property

Who owns the finished work?

Can either party reuse it?

Does ownership transfer after payment?

Confidentiality

What information must remain private?

How long do those obligations last?

Dispute Resolution

Will disagreements go to court?

Arbitration?

Mediation?

These sections often receive less attention than they deserve.

AI Makes Contract Review Faster

AI has become surprisingly good at helping people review documents.

It can:

Summarize lengthy contracts
Explain unfamiliar legal terms
Highlight unusual clauses
Compare different versions of an agreement
Identify missing definitions or inconsistent language

That can save hours of reading.

But AI Doesn't Know Your Business

Here's where many people make a mistake.

AI can explain what a clause says.

It cannot tell you whether that clause is acceptable for your business goals.

For example, an unlimited liability clause might be perfectly reasonable for one company and completely unacceptable for another.

That decision depends on your industry, your leverage in the negotiation, and your tolerance for risk.

Context matters.

Build a Simple Review Checklist

Before signing any agreement, ask yourself:

What am I promising?
What is the other party promising?
What happens if either side fails?
Can this agreement be terminated easily?
Who carries the biggest financial risk?
Is anything unclear enough to cause a dispute later?

These questions often reveal issues that are easy to miss during a quick read.

Better Decisions Start With Better Understanding

You don't need a law degree to read contracts more effectively.

You need curiosity, patience, and the habit of looking beyond the obvious.

AI can help make contracts easier to understand.

Legal professionals help determine whether the terms actually protect your interests.

Using both together leads to better decisions than relying on either one alone.

Final Thoughts

Contracts are not just paperwork.

They're tools for managing risk.

The more you understand what's inside them before you sign, the fewer surprises you'll face later.

Whether you're reviewing a client agreement, hiring a contractor, or negotiating a partnership, taking the time to understand the risks today can prevent costly problems tomorrow.

AI Can Read a Contract in Seconds. Should You Trust It?

James Glaser — Fri, 24 Jul 2026 19:22:57 +0000

AI can summarize a 30-page contract in seconds.

It can point out unusual clauses, explain legal terms in plain English, and even compare two versions of an agreement.

That's impressive.

But should you trust it?

The answer isn't as simple as yes or no.

AI Is Great at Explaining, Not Deciding

One of AI's biggest strengths is turning complex legal language into something most people can understand.

Instead of reading this:

"The indemnifying party shall defend, indemnify, and hold harmless..."

An AI can explain:

"If something goes wrong because of your actions, you may have to cover the other party's losses."

For many people, that alone makes legal documents far less intimidating.

But understanding a clause and deciding whether to agree to it are two very different things.

Context Changes Everything

A non-compete agreement might seem reasonable for one employee and completely unreasonable for another.

A contract that looks standard in one state could be unenforceable in another.

An AI can explain what a clause usually means.

It cannot fully understand your business, your goals, or every legal nuance that applies to your situation.

That's where human judgment matters.

AI Never Gets Tired

One advantage AI has over humans is consistency.

It won't lose focus after reviewing the tenth contract of the day.

It won't skip a paragraph because it's rushing to another meeting.

It can quickly identify:

Missing signatures
Undefined terms
Inconsistent dates
Duplicate clauses
Missing sections

That makes it a valuable first reviewer.

The Best Workflow Isn't AI or Lawyers

It's both.

Imagine this process:

Upload a contract.
AI summarizes the document.
AI highlights unusual clauses.
A lawyer reviews the important sections.
The client receives both a plain English explanation and professional legal advice.

Everyone saves time without sacrificing quality.

AI Should Help You Ask Better Questions

Many people think AI should provide legal answers.

A better goal is helping people ask better questions.

Instead of replacing legal professionals, AI can help users understand what they're reading before they speak with one.

That often leads to more productive conversations and better decisions.

The Future of Legal AI

Legal AI isn't about replacing attorneys.

It's about making legal information more accessible.

The more people understand contracts, policies, and legal documents, the more confident they become when making important decisions.

AI can make the law easier to understand.

People still make the decisions.

Final Thoughts

The biggest value of AI in legal work isn't speed.

It's clarity.

When AI helps people understand complex legal information without pretending to replace professional advice, everyone benefits.

Technology works best when it empowers people, not when it tries to replace them.