DEV Community

Filippo
Filippo

Posted on AI-assisted

How do you measure whether a repository is ready for AI contributors?

A repository can look healthy to a human contributor while leaving an AI coding agent without clear operating rules.

A README may explain the project. CONTRIBUTING.md may explain how to submit a pull request. CI may verify the result.

But several questions can remain unanswered:

  • Is AI-assisted contribution permitted?
  • Must generated work be disclosed?
  • Which files or systems may an agent modify?
  • Which checks must run before submission?
  • Which actions require explicit human approval?
  • What should happen when instructions conflict?
  • Who owns and maintains these rules?

That is a different problem from conventional repository health.

A deterministic approach

I built RepoPolicyScore to evaluate this problem using published, deterministic checks:

https://repopolicyscore.com

The current scoring model contains 25 checks grouped into seven areas:

  1. Discoverability of contribution rules
  2. AI-use expectations
  3. Human approval and review responsibility
  4. Agent boundaries
  5. Validation and required checks
  6. Risky actions, exceptions and escalation
  7. Policy ownership and maintenance

The same repository evidence evaluated with the same scoring version produces the same result.

An LLM does not decide whether a repository “feels” ready.

That distinction matters because maintainers need findings they can inspect, reproduce and challenge.

AI readiness is not repository health

RepoPolicyScore keeps AI Contribution Readiness and Repository Health separate.

Repository Health examines operational signals such as:

  • project documentation
  • licensing
  • contribution guidance
  • CODEOWNERS
  • issue and pull-request templates
  • CI automation
  • security guidance
  • dependency-maintenance automation
  • recent maintenance activity

These signals matter, but they do not prove that an AI coding agent understands its boundaries.

Combining everything into one score would hide that distinction.

Findings should lead to remediation

Identifying a missing rule is only useful if the maintainer can close the gap.

RepoPolicyScore therefore provides evidence behind its findings, prioritizes fixes and can generate repository-specific policy wording.

The intended workflow is:

  1. Scan a public GitHub repository.
  2. Inspect the findings and supporting evidence.
  3. Generate policy wording for the identified gaps.
  4. Copy or download the Markdown.
  5. Commit it to the repository.
  6. Rescan and verify the change.

The scanner is read-only. It requires no account or OAuth, doesn’t clone the repository, doesn’t execute its code and doesn’t request write access.

Where I need maintainer feedback

The product is live, but the scoring model needs exposure to more real repository structures.

The most useful feedback is not whether the page looks good. It is:

  • false positives
  • false negatives
  • valid policy structures the scanner misses
  • checks that deserve different weighting
  • unclear or impractical remediation wording
  • important AI-contribution rules missing from the model

If you maintain a public GitHub repository, scan it and challenge the result:

https://repopolicyscore.com

I’d rather find weaknesses in the model now than preserve a score that maintainers cannot trust.

Top comments (0)