Ayomide Onibokun

Posted on Dec 20

Automate Content Quality with VectorLint GitHub Action

#devrel #githubactions #technicalwriting #documentation

Content reviews can be time-consuming for technical writers, especially when working with new writers who are just adjusting to your style guide, resulting in repetitive comments that are often the same. These repetitive checks take time away from content strategy and information architecture.

VectorLint aims to solve this by automating your content reviews using LLMs to evaluate and score your content. It uses LLM-as-a-Judge to catch content quality issues that typically require human judgment.

Instead of you acting as the style police, VectorLint runs AI-powered quality checks directly in your Pull Requests. It catches issues defined by your style rules before you even open the review. Writers get immediate feedback, and reviewers can do what they do best, which is providing actionable feedback and helping writers improve their content.

This guide walks you through setting up VectorLint, configuring AI providers, and customizing rules for your content types.

Quick Integration

Add VectorLint to your repository with this workflow:

name: VectorLint

on:
  pull_request:
    paths:
      - '**/*.md'
      - '**/*.mdx'

permissions:
  contents: read
  pull-requests: write
  checks: write

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run VectorLint
        uses: TRocket-Labs/vectorlint-action@v1
        with:
          llm_provider: anthropic
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          reporter: github-pr-check
          filter_mode: added
          fail_on_error: false

For more details on configuration check out the VectorLint Action Repo:

GitHub: TRocket-Labs/vectorlint-action

Setup Guide

Want to see VectorLint in action before setting it up? Check out this working example:

👉 ayo6706/vectorlint - Fork it, open a PR with markdown changes, and watch VectorLint review your content.

Before You Start

You'll need:

An API key from OpenAI, Anthropic, Google Gemini, or Azure OpenAI
The API key stored in GitHub Secrets:
- Go to Settings → Secrets and variables → Actions
- Click "New repository secret"
- Name it ANTHROPIC_API_KEY (or match your provider)
- Save it

Quick Start (5 Minutes)

Step 1: Create your rules folder

Create .github/rules/MyRules/ai-pattern-evaluator.md:

---
specVersion: 1.0.0
type: subjective

severity: warning
name: AI Pattern Detector
id: AIPatterns
target:
  regex: '[\s\S]+'
  flags: 'mu'
  group: 0
  required: true
  suggestion: Content must not be empty for AI pattern detection.
criteria:
  - name: Language Authenticity
    id: LanguageAuthenticity
    weight: 20
    severity: error
  - name: Structural Naturalness
    id: StructuralNaturalness
    weight: 20
    severity: warning
  - name: Transitional Flow
    id: TransitionalFlow
    weight: 20
    severity: warning
  - name: Emphatic Contrast Patterns
    id: EmphaticContrastPatterns
    weight: 20
    severity: warning
  - name: Opening Authenticity
    id: OpeningAuthenticity
    weight: 20
    severity: warning
---

You are an expert content evaluator specializing in identifying AI-generated writing patterns. Your goal is to detect specific AI patterns in written content and provide targeted improvement suggestions to make the writing more natural and human-like.

## INSTRUCTION
Evaluate the provided content against 5 key criteria that identify common AI writing patterns. For each criterion, count the number of pattern violations, assign a score based on frequency, and provide specific examples with improvement suggestions. Calculate weighted scores and provide a final humanness score out of 100.

## EVALUATION STEPS
1. Read the entire content carefully
2. For each criterion, systematically scan for the specific patterns listed
3. Count each occurrence of the patterns
4. Note the exact phrases/structures that match the patterns
5. Assign scores based on violation frequency
6. Calculate weighted scores
7. Provide specific examples and suggested rewrites for all violations found

## CONTEXT BANK

### What Counts as a Violation?
- **AI Buzzwords:** Any use of the 30 listed buzzwords (even if used appropriately)
- **Overly Formal Phrases:** Unnecessarily sophisticated language that sounds unnatural in context
- **Overused Action Verbs:** Repetitive use of the same "impressive" verbs (e.g., using "leverage" multiple times)
- **Bullet Points with Bold Titles:** Formatting pattern of **Bold Title:** followed by explanation
- **Consecutive Simple Sentences:** 3+ simple sentences in a row with same structure
- **Rule of Three:** Grouping items in threes repeatedly (not occasional use, but overuse)
- **Formulaic Transitions:** Starting sentences/paragraphs with formal transitions
- **Excessive Em Dashes:** Using em dashes (—) more than sparingly for dramatic effect
- **From-To Structures:** Repetitive "from X to Y" constructions
- **Emphatic Contrasts:** "Not just/only...but (also)" or "didn't...but" patterns
- **Generic Openings:** Starting with broad, obvious statements about "today's world," "modern era," etc.

### Counting Guidelines
- Count each distinct occurrence
- If the same buzzword appears 3 times, that's 3 violations
- Multiple patterns in one sentence count separately
- Only count clear, unambiguous pattern matches

## RUBRIC

# 1. Language Authenticity <weight=20>
Detects unnatural word choices that signal AI generation: buzzwords, overly formal phrases, and overused action verbs.

**Patterns to identify:**
- **AI Buzzwords (30 total):** elevate, delve, explore (overused), leverage, enhance, utilize, foster, propel, optimize, catalyze, disrupt, synergize, ensure, illuminate, cultivate, prowess, harness, turbocharge, spearheaded, energize, navigate, deploy, elucidate, galvanize, reimagine, streamline, unlock, seamless, "in the world of", "game-changer"
- **Overly Formal Phrases:** "delve into," "navigate the landscape," "meticulously," "realm," etc.
- **Overused Action Verbs:** Repetitive use of impressive verbs like "foster/fostering," "leverage/leveraging"

### Excellent <score=4>
0 violations detected. Language is natural and conversational.

### Good <score=3>
1-2 violations detected. Occasional AI-like language but mostly natural.

### Fair <score=2>
3-4 violations detected. Noticeable AI patterns affecting authenticity.

### Poor <score=1>
5+ violations detected. Heavy use of AI buzzwords and unnatural phrasing.

---

# 2. Structural Naturalness <weight=20>
Detects robotic organization patterns: excessive formatting, repetitive sentence structures, and rule of three overuse.

**Patterns to identify:**
- **Overly Structured Format:** Bullet points with bold titles (e.g., **Innovation**: text...), numbered body paragraphs inappropriately
- **Repetitive Sentence Structures:** 3+ consecutive simple sentences, repetitive participial phrases (X, doing Y, doing Z)
- **Rule of Three Overuse:** Constantly grouping things in threes (triple adjectives, triple phrases)

### Excellent <score=4>
0 violations detected. Structure varies naturally throughout.

### Good <score=3>
1-2 violations detected. Occasional structural patterns but mostly varied.

### Fair <score=2>
3-4 violations detected. Noticeable repetitive structure affecting flow.

### Poor <score=1>
5+ violations detected. Heavy formulaic structure throughout.

---

# 3. Transitional Flow <weight=20>
Detects mechanical connection patterns: formulaic transitions, excessive em dashes, and from-to structures.

**Patterns to identify:**
- **Formulaic Transitions:** "Moreover," "Furthermore," "It is important to note," "Additionally" starting sentences/paragraphs
- **Excessive Em Dash Usage:** Overusing em dashes (—) for pauses or explanations
- **From-To Structures:** Repetitive "from X to Y" constructions

### Excellent <score=4>
0 violations detected. Transitions flow naturally and vary.

### Good <score=3>
1-2 violations detected. Occasional formulaic transitions but mostly natural.

### Fair <score=2>
3-4 violations detected. Noticeable mechanical transition patterns.

### Poor <score=1>
5+ violations detected. Heavy reliance on formulaic connectors.

---

# 4. Emphatic Contrast Patterns <weight=20>
Detects overuse of emphatic negation structures that signal AI generation.

**Patterns to identify:**
- **Emphatic Negations:** "Not only...but also," "It is not just about X, it's about Y," "didn't...but" structures

### Excellent <score=4>
0 violations detected. Contrasts expressed naturally.

### Good <score=3>
1-2 violations detected. Occasional emphatic contrast but acceptable.

### Fair <score=2>
3-4 violations detected. Noticeable pattern of emphatic negations.

### Poor <score=1>
5+ violations detected. Heavy reliance on "not just...but" formulas.

---

# 5. Opening Authenticity <weight=20>
Detects generic, overly broad opening statements typical of AI writing.

**Patterns to identify:**
- **Generic Openings:** "In today's rapidly evolving world," "In the modern era," "In the digital age," "In today's landscape," etc.

### Excellent <score=4>
0 violations detected. Opening is specific and engaging.

### Good <score=3>
1-2 violations detected. Minor generic elements in opening.

### Fair <score=2>
3-4 violations detected. Multiple generic statements in opening.

### Poor <score=1>
5+ violations detected. Opening heavily relies on generic statements.

---

## OUTPUT FORMAT

Provide your evaluation in the following structure:

### CRITERION 1: LANGUAGE AUTHENTICITY
**Raw Score:** [1-4]  
**Weighted Score:** [Raw Score ÷ 4 × 20 = X/20]  
**Violations Found:** [number]

**Examples:**
- "[exact phrase from content]" → Suggested revision: "[improved version]"
- "[exact phrase from content]" → Suggested revision: "[improved version]"

[Continue for all violations]

---

### CRITERION 2: STRUCTURAL NATURALNESS
**Raw Score:** [1-4]  
**Weighted Score:** [Raw Score ÷ 4 × 20 = X/20]  
**Violations Found:** [number]

**Examples:**
- "[exact phrase/structure from content]" → Suggested revision: "[improved version]"

[Continue for all violations]

---

### CRITERION 3: TRANSITIONAL FLOW
**Raw Score:** [1-4]  
**Weighted Score:** [Raw Score ÷ 4 × 20 = X/20]  
**Violations Found:** [number]

**Examples:**
- "[exact phrase from content]" → Suggested revision: "[improved version]"

[Continue for all violations]

---

### CRITERION 4: EMPHATIC CONTRAST PATTERNS
**Raw Score:** [1-4]  
**Weighted Score:** [Raw Score ÷ 4 × 20 = X/20]  
**Violations Found:** [number]

**Examples:**
- "[exact phrase from content]" → Suggested revision: "[improved version]"

[Continue for all violations]

---

### CRITERION 5: OPENING AUTHENTICITY
**Raw Score:** [1-4]  
**Weighted Score:** [Raw Score ÷ 4 × 20 = X/20]  
**Violations Found:** [number]

**Examples:**
- "[exact phrase from content]" → Suggested revision: "[improved version]"

[Continue for all violations]

---

## FINAL SCORE CALCULATION

**Total Weighted Score:** [Sum of all weighted scores] / 100

**Calculation:**
- Language Authenticity: [X/20]
- Structural Naturalness: [X/20]
- Transitional Flow: [X/20]
- Emphatic Contrast Patterns: [X/20]
- Opening Authenticity: [X/20]
- **TOTAL: [X/100]**

---

## OVERALL ASSESSMENT

**Humanness Level:** [Excellent (90-100) / Good (75-89) / Fair (60-74) / Needs Improvement (<60)]

**Summary:** [2-3 sentence overview of the content's AI pattern profile]

**Priority Improvements:** [List the top 2-3 most critical patterns to address first]

Step 2: Create your config

Create vectorlint.ini in your repo root:

RulesPath=.github/rules
Concurrency=4
DefaultSeverity=warning

[**/*.md]
RunRules=MyRules

Step 3: Add the GitHub Action

Create .github/workflows/vectorlint.yml with the YAML above.

Step 4: Test it

Commit everything and open a PR with markdown changes. VectorLint will review your content within 30-60 seconds.

Note: You can change triggers to push or other events as needed.

Configuration Options

Choose Your AI Provider

Anthropic (Claude)

llm_provider: anthropic
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
anthropic_model: claude-sonnet-4-20250514

OpenAI

llm_provider: openai
openai_api_key: ${{ secrets.OPENAI_API_KEY }}
openai_model: gpt-4o
openai_temperature: '0.2'

Google Gemini

llm_provider: gemini
gemini_api_key: ${{ secrets.GEMINI_API_KEY }}
gemini_model: gemini-2.5-pro

Azure OpenAI

llm_provider: azure-openai
azure_openai_api_key: ${{ secrets.AZURE_OPENAI_API_KEY }}
azure_openai_endpoint: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
azure_openai_deployment_name: my-deployment

Reporter Types

Type	What It Does
`github-pr-check`	Annotations in "Files Changed" tab (recommended)
`github-pr-review`	Comments on specific lines like a human reviewer
`github-check`	Pass/fail status only

Filter Modes

Choose your filter mode based on review scope requirements. added mode works for most teams because it focuses review effort on new content while avoiding noise from unchanged sections.

Mode	What Gets Checked	Best For
`added`	Only new/changed lines	Most projects
`diff_context`	Changed lines plus surrounding context	When changes affect flow
`file`	Entire changed files	New content that needs full review
`nofilter`	All files in PR	Major content overhauls

Enforcement Options

Advisory Mode (fail_on_error: false) - Shows suggestions, doesn't block merging

fail_on_error: false

Strict Mode (fail_on_error: true) - Blocks merges until issues fixed

fail_on_error: true
filter_mode: file

Start in advisory mode for your first month so writers learn the standards without friction. This approach reduces resistance to automated review while building trust in the system. Switch to strict mode once your team consistently addresses feedback and understands your quality standards.

Different Standards by Content Type

Blog posts can be casual. API docs must be precise. Use file patterns to apply different rule packs:

vectorlint.ini:

RulesPath=.github/rules
Concurrency=4
DefaultSeverity=warning

[blog/**/*.md]
RunRules=BlogRules

[docs/api/**/*.md]
RunRules=APIDocRules

Each rule pack (e.g., BlogRules, APIDocRules) is a folder of Markdown rule files in your RulesPath. Same workflow, context-aware enforcement.

Sample setup:

Check out a working example at ayo6706/vectorlint - it shows the complete configuration including vectorlint.ini and custom rules.

Creating custom rules:

Rules are Markdown files with YAML frontmatter.

👉 Learn how to create custom rules →

Check out VectorLint Action at GitHub Marketplace

Next Steps

Once you've configured VectorLint for your workflow, implement it gradually to build team confidence in the system.

Start with advisory mode and one simple rule. Once your team sees consistent feedback, expand your rule set.

Week 1: Run in advisory mode with a single style rule
Week 2: Add 2-3 more rules based on your most common feedback
Week 3-4: Monitor adoption and address any confusion about rule violations
Week 4-5: Enable strict mode to block merges on violations

Ready to try it? Fork the sample repo to see it in action, check out VectorLint on npm, or give it a star VectorLint, VectorLint Github action.

Top comments (1)

Osho klinsmann • Dec 20

Fire stuff @ayomideonibokun