DEV Community

Cover image for My AI Side Project Would Fail an EU AI Act Audit — Here's How I Fixed It
Arkforge
Arkforge

Posted on

My AI Side Project Would Fail an EU AI Act Audit — Here's How I Fixed It

I've been building a small AI app — a LangChain-powered tool that summarizes legal documents. Nothing fancy, just a side project I was tinkering with on weekends.

Then someone in a GitHub discussion asked me: "Is your app EU AI Act compliant?"

My honest answer was "I'm too small for that to matter." Turns out, I was wrong.

The EU AI Act doesn't care about your company size

The regulation that started applying in February 2025 (with full enforcement coming August 2026) covers any AI system made available in the EU market. That includes:

  • Your SaaS with 10 users
  • Your open-source tool on GitHub
  • Your internal tool if it processes EU citizen data

There's no "small developer exemption" for the core transparency requirements. If you're using an AI model in production, you have obligations.

I know because I read the actual regulation text (all 144 pages — don't recommend it on a Friday evening).

I scanned my own project

I pointed a compliance scanner at my own repo. Here's what my project looked like:

├── app.py              # LangChain pipeline
├── requirements.txt    # langchain, openai
├── prompts/
│   └── summarizer.py   # System prompts
└── README.md           # "A tool that summarizes stuff"
Enter fullscreen mode Exit fullscreen mode

The scan detected:

  • LangChain framework (via requirements.txt)
  • OpenAI API usage (via imports in app.py)
  • Risk category: Limited (text generation system)

And here's what I was missing — three things that took me a total of 35 minutes to fix.

1. No transparency disclosure

Article 50 of the EU AI Act requires that users know when they're interacting with AI-generated content. My app returned summaries with zero indication they came from a machine.

Before:

def summarize(document: str) -> str:
    return chain.invoke({"document": document})
Enter fullscreen mode Exit fullscreen mode

After:

from datetime import datetime

def summarize(document: str) -> dict:
    result = chain.invoke({"document": document})
    return {
        "summary": result,
        "ai_disclosure": "This summary was generated by an AI system (OpenAI GPT-4 via LangChain).",
        "model": "gpt-4",
        "generated_at": datetime.utcnow().isoformat()
    }
Enter fullscreen mode Exit fullscreen mode

Time: 5 minutes. The API response now tells consumers exactly what generated the content.

2. No technical documentation

Even for limited-risk systems, having documentation is the difference between "we take compliance seriously" and "we'll figure it out when the auditor shows up."

I had a 3-line README. Here's what I added as AI_COMPLIANCE.md:

## AI System Documentation

- **Purpose**: Summarize legal documents for quick review
- **Model**: OpenAI GPT-4 via LangChain
- **Training data**: None (uses pre-trained model via API)
- **Risk category**: Limited (AI-generated text, Article 50)
- **Transparency**: All outputs include AI disclosure
- **Limitations**: May miss nuances in complex legal language.
  Not suitable for legal advice.
- **Human oversight**: Summaries are review assistance only
- **Data retention**: No user data stored beyond session
Enter fullscreen mode Exit fullscreen mode

Time: 20 minutes (mostly thinking about what "limitations" to be honest about).

3. No input/output logging

Not a strict legal requirement for limited-risk systems, but I realized I had no way to trace what went in and what came out. When you're building with LLMs, auditability isn't just compliance — it's debugging.

import logging
import uuid

logger = logging.getLogger("ai_audit")

def summarize_with_audit(document: str) -> dict:
    request_id = uuid.uuid4().hex[:8]
    logger.info(f"[{request_id}] Input length: {len(document)} chars")

    result = chain.invoke({"document": document})

    logger.info(f"[{request_id}] Output length: {len(result)} chars")
    return {
        "summary": result,
        "request_id": request_id,
        "ai_disclosure": "Generated by AI (OpenAI GPT-4)"
    }
Enter fullscreen mode Exit fullscreen mode

Time: 10 minutes. Now every request has a trace.

What surprised me

The biggest surprise wasn't what I was missing — it was how easy the fixes were. 35 minutes total.

The hard part was knowing I needed to do it. That's the real gap for indie developers: not technical difficulty, but awareness. The EU AI Act is 144 pages of legal text, and the practical developer implications are buried deep.

My other surprise: my project was "limited risk," not "high risk." Most developer tools fall into this category. The obligations are real but manageable — mainly transparency and documentation. High-risk (medical, hiring, law enforcement) is where it gets heavy.

How I automated it

I now run a basic compliance check in CI. Here's the GitHub Action:

# .github/workflows/ai-compliance.yml
name: EU AI Act Compliance Check

on: [push, pull_request]

jobs:
  compliance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Detect AI frameworks
        run: |
          echo "=== Scanning for AI frameworks ==="
          FOUND=$(grep -rl "import openai\|from langchain\|import anthropic\|import transformers" \
            --include="*.py" . 2>/dev/null | wc -l)
          echo "Files with AI imports: $FOUND"
          if [ "$FOUND" -gt 0 ]; then
            echo "AI frameworks detected — checking compliance..."
          fi

      - name: Verify transparency disclosure
        run: |
          DISCLOSURES=$(grep -rl "ai_disclosure\|AI-generated\|generated by AI" \
            --include="*.py" . 2>/dev/null | wc -l)
          if [ "$DISCLOSURES" -eq 0 ]; then
            echo "::warning::No AI transparency disclosure found in code"
          else
            echo "Transparency disclosures found in $DISCLOSURES files"
          fi

      - name: Check compliance documentation
        run: |
          if [ ! -f "AI_COMPLIANCE.md" ]; then
            echo "::warning::No AI_COMPLIANCE.md found"
            echo "Consider adding AI system documentation"
          else
            echo "AI compliance documentation found"
          fi
Enter fullscreen mode Exit fullscreen mode

This catches the obvious things. For a deeper scan that detects 16 AI frameworks and maps them to specific EU AI Act obligations, I use this free MCP compliance scanner.

What I'd tell past me

  1. Add AI_COMPLIANCE.md on day 1 — it takes 20 minutes and forces you to think about what you're building
  2. Tag every AI outputai_disclosure in your response schema is the easiest obligation to meet
  3. Know your risk category — most side projects are "limited" or "minimal," which means lighter obligations. Don't panic.
  4. Log request IDs — not just for compliance, but because debugging LLM outputs without traces is suffering

The EU AI Act isn't trying to kill indie projects. It's mostly about making sure people know when AI is involved in decisions that affect them. That's a bar most of us can clear with an afternoon of work.

The enforcement deadline is August 2026. You have time — but starting now means you won't scramble later.

Top comments (0)