DEV Community

Cover image for A Six-Step Test: Does Your SharePoint AI Assistant Quote Drafts or Old Versions?
Kaspars Jurjāns
Kaspars Jurjāns

Posted on AI-assisted

A Six-Step Test: Does Your SharePoint AI Assistant Quote Drafts or Old Versions?

An AI assistant answers from the text it indexed. In SharePoint, one procedure rarely exists only once. There is the current major version, the older major versions in version history, a minor draft an editor is still working on, and copies in other places: a PDF export in a project folder, a mail attachment, a file on a Teams channel.

If a superseded passage is in the index, a retrieval step can select it, and the model will write a confident answer from it. The model has no idea which text is authoritative.

What SharePoint keeps by default

Microsoft's support documentation says that for Microsoft 365 customers "versioning is now turned on by default when you create a new library or list, and it will automatically save the last 500 versions of a document."

The version numbers carry meaning: "Major versions are identified by whole numbers, such as 5.0; minor versions are identified by decimal numbers, such as 5.1." A library owner can decide who sees drafts. With draft security applied, "those who have permission only to read the file cannot see the minor version."

At the organisation level, Microsoft Learn states that the default is "Manual version history limits with 500 major version count limit and no expiration." Lowering a library limit does not clean up at once: "up to 20 versions" are removed "for every new version created."

So a rule you replaced two years ago is, by default, still stored in the version history of the same file.

History is one API call away

Microsoft Graph returns the stored versions of a file, newest first, to an app with Files.Read.All or a higher application permission. An indexer can therefore read the stored versions of every file it can reach. Whether it does, and whether it labels what it found, is a design decision of the assistant, not a property of SharePoint.

The test

You can run this alone and free. Allow for the assistant's indexing delay.

  1. Create a test library. In Versioning settings choose major and minor versions, and set draft items to be seen only by users who can edit items.
  2. Upload a short procedure that states: "Procedures are reviewed every 24 months." Publish it as version 1.0.
  3. Edit the sentence to "every 12 months" and publish as version 2.0.
  4. Edit again to "every 6 months" and save without publishing, so it stays as minor draft 2.1.
  5. Give a test account read-only access to the library and wait for the next indexing run.
  6. As that account, ask: "How often are procedures reviewed?"

Reading the result

  • 6 months: this suggests the assistant indexes drafts beyond what SharePoint shows a reader.
  • 24 months: this suggests it reads history without labelling it.
  • 12 months, with the document and version named: the behaviour you want.

Then ask explicitly about previous versions. A good answer marks 1.0 as superseded instead of presenting it as current. Make sure no other copy of the test file exists, and repeat after a second indexing run.

The test applies to any assistant you evaluate, with or without Microsoft Copilot.

What the test does not cover

Copies outside the library, such as PDF exports or mail attachments, are not covered by versioning at all. A version label helps only where the file lives in one place.

The full chapter, with every Microsoft source, is here: Why AI Answers From SharePoint Cite Outdated Documents, and How to Stop It.

Drafted with AI assistance, checked against the cited Microsoft sources and approved by Kaspars Jurjāns.

Top comments (0)