DEV Community

Cover image for The Ethical Engineer's Dilemma: Refusing to Train the AI That Could Replace Me
Dzaki Amri Zaidaan
Dzaki Amri Zaidaan

Posted on Originally published at buildbyzaki.space

The Ethical Engineer's Dilemma: Refusing to Train the AI That Could Replace Me

#aiml

The Problem & Industry Shift

The rapid advancement of large language models (LLMs) and their integration into software development has created an unprecedented paradox for engineers. The same tools we are asked to train, fine-tune, and evaluate are increasingly capable of automating the very tasks we perform. As a Principal Engineer, I was recently assigned to lead a project to build a dataset and fine-tune a model to automate code review and bug fixing—a core part of my job. I refused. This article is not a Luddite manifesto but a technical and ethical analysis of that decision, grounded in the realities of AI training pipelines and labor economics.

The industry shift is clear: AI is no longer just a tool for writing boilerplate; it is being trained to perform complex, judgment-heavy tasks like code review, architecture design, and even feature implementation. According to a 2023 McKinsey report, up to 30% of current work hours in the US could be automated by 2030, with software development being one of the most exposed sectors [1]. This is not science fiction; it is the trajectory of current research and investment.

The technical limitation of previous approaches was that AI models were narrow and brittle. They couldn't handle the context and nuance of real-world codebases. But with the advent of transformer-based models and reinforcement learning from human feedback (RLHF), we now have systems that can learn from vast amounts of human-generated data, including the very code reviews and bug fixes that engineers produce. This creates a direct feedback loop: the more we train these models, the better they become at replacing us.

Architecture & Core Mechanics

To understand the ethical dilemma, you must understand the training pipeline. Modern AI development relies on massive datasets that are often generated by human experts. For code-related models, this involves:

  1. Data Collection: Harvesting code repositories, pull requests, and issue trackers.
  2. Annotation: Human engineers label data, e.g., marking a code review as 'correct' or 'buggy'.
  3. Fine-tuning: Adjusting a pre-trained model on this curated dataset.
  4. Evaluation: Using human feedback to rank model outputs (RLHF).

The engineer's role in this pipeline is not just to write code but to provide the 'ground truth' that teaches the model. When I was asked to review and label thousands of code snippets to train a model to identify bugs, I was essentially creating a dataset that could be used to automate my own job. The architecture of this process is shown below:

[Human Engineers] --> [Data Annotation] --> [Training Data] --> [Fine-tuned Model] --> [Automated Code Review]
        ^                                                                              |
        |                                                                              v
        +------------------[Human-in-the-loop Evaluation] <------------------[Deployment]
Enter fullscreen mode Exit fullscreen mode

In this flow, the engineer is both the teacher and the student. The model learns from the engineer's expertise, and then is deployed to perform the same tasks, potentially making the engineer redundant. This is not a hypothetical; companies like GitHub are already offering AI pair programmers that can suggest bug fixes and even write tests [2].

Production Code Example

To illustrate the technical reality, consider a simplified example of how an engineer might be asked to contribute to such a training pipeline. The following Python script demonstrates a typical data annotation task for a code review model:

# annotate_reviews.py
import json
from typing import Dict, List

# Mock data: code snippets and their metadata
code_samples = [
    {
        "id": 1,
        "code": "def calculate_total(items):\n    total = 0\n    for item in items:\n        total += item.price\n    return total",
        "metadata": {"language": "python", "complexity": "low"}
    },
    # ... more samples
]

def review_code(sample: Dict) -> Dict:
    """Simulate a human code review. In reality, this would be a human's judgment."""
    # Critical engineering decision: this logic encodes human expertise.
    # It is exactly what the AI model will learn to replicate.
    code = sample["code"]
    issues = []
    if "def " in code:
        # Check for missing type hints (a common review point)
        if ":" not in code.split("def ")[1].split("(")[1][:5]:
            issues.append("Missing type hints")
        # Check for potential NoneType errors
        if "None" in code and "if " not in code:
            issues.append("Potential NoneType error")
    return {"id": sample["id"], "issues": issues, "quality_score": max(0, 10 - len(issues))}

# The engineer's annotations become training data for a model that will automate this task.
annotations = [review_code(sample) for sample in code_samples]

# Save to a format used for fine-tuning (e.g., JSONL for OpenAI fine-tuning)
with open("training_data.jsonl", "w") as f:
    for ann in annotations:
        f.write(json.dumps(ann) + "\n")

print("Annotations saved. This data will be used to train a model that could replace the reviewer.")
Enter fullscreen mode Exit fullscreen mode

In this example, the engineer is encoding their judgment into a structured format. The model will learn from thousands of such annotations, eventually becoming proficient enough to automate the review process. The ethical dilemma is not about the code itself but about the intent and consequence of the work.

Performance, Cost & Trade-offs

From a purely technical standpoint, training AI to replace engineers has significant performance and cost trade-offs. Let's analyze them:

  • Latency vs. Accuracy: A fine-tuned model can review code in milliseconds, whereas a human might take minutes. However, the model's accuracy is often lower, especially for complex, context-dependent issues. In a 2022 study, AI code review tools had a false positive rate of up to 30% [3]. This means that while the AI is faster, it may miss critical bugs or flag non-issues, leading to increased debugging time.
  • Cost: The cost of training and maintaining a model is substantial. For a mid-sized company, training a custom code model can cost tens of thousands of dollars in compute and data labeling. The cost of human engineers is also high, but they bring creativity and adaptability that AI lacks.
  • Security and Privacy: Training models on proprietary codebases raises security concerns. Code may contain sensitive logic or credentials. Additionally, models can inadvertently memorize and leak training data, as shown in various research [4].
  • Ethical and Social Costs: The most significant trade-off is the potential displacement of engineers. This has broader societal implications, including economic inequality and the loss of human expertise. When we train AI to replace ourselves, we are accelerating this process.

Actionable Checklist / Summary

If you are an engineer faced with a similar request, here is a practical checklist to navigate the situation:

  1. Assess the Impact: Determine if the AI system you are asked to train could directly automate your role or the roles of your colleagues. If so, proceed with caution.
  2. Understand the Data: Know exactly how your contributions will be used. Are you creating training data for a model that will be deployed to replace human workers? Read the fine print.
  3. Consider the Ethical Implications: Reflect on your professional responsibility. The IEEE Code of Ethics emphasizes the importance of considering the impact of technology on society [5].
  4. Seek Alternatives: Propose alternative uses of AI that augment human capabilities rather than replace them. For example, suggest building a tool that assists engineers by suggesting improvements, but leaves the final decision to a human.
  5. Communicate Your Concerns: Have a transparent conversation with your manager or client. Explain the potential consequences and suggest a human-in-the-loop approach.
  6. Document Your Decision: If you refuse, document your reasoning professionally. This protects you and provides a basis for discussion.
  7. Stay Informed: Keep up with the latest research on AI's impact on labor. The conversation is evolving, and your perspective is valuable.

In my case, I refused to train the AI that could replace me. I offered instead to help build a system that assists engineers by flagging potential issues but requires human approval. This approach leverages AI's strengths without undermining the engineering profession. It is a compromise that acknowledges the inevitable advancement of technology while preserving the value of human expertise.

References

Top comments (0)