Most of us have seen AI agents complete impressive tasks. They write code, analyze documents, generate reports, and solve problems that would have been difficult to automate just a few years ago.
But there's a question I keep coming back to.
What happens when we stop assigning AI individual tasks and instead give it an entire responsibility?
Not a five-minute benchmark. Not a carefully prepared demonstration.
A responsibility that requires sustained competence, adapting to unexpected circumstances, and knowing what to do when the instructions no longer fit the situation.
I've started an open-source research initiative called Multa Lumina to explore that question.
And I'd like to invite other developers, researchers, and curious people to investigate it with me.
Two challenges, one research objective
Multa Lumina is organized around two very different tests of autonomous intelligence.
Challenge 1: The Full Job Test
Imagine an AI system joining an unfamiliar company as a remote employee.
It might work as a software engineer, customer success representative, or financial analyst.
To succeed, it must independently:
- Complete ordinary onboarding and learn the company's procedures.
- Understand documentation and carry out assigned responsibilities.
- Communicate appropriately with coworkers.
- Adapt when policies, priorities, or instructions change.
- Recognize and resolve unexpected problems.
- Maintain acceptable performance over weeks or months.
It cannot rely on undisclosed human operators or specialized retraining for that particular employer.
And its work must deliver genuine economic value.
Completing a hundred isolated tasks isn't necessarily equivalent to performing a job. I want to investigate the difference.
Challenge 2: The Wozniak Coffee Test
Steve Wozniak proposed a fascinating challenge involving physical intelligence.
Can an AI-powered robot enter an unfamiliar American home, locate the kitchen, identify the available equipment and ingredients, and prepare coffee without assistance?
Think about everything that seemingly simple task involves.
The robot must recognize unfamiliar objects, navigate an unpredictable environment, understand how different appliances work, manipulate physical objects, and adapt to circumstances it hasn't rehearsed.
A successful demonstration would need to take place safely, with the homeowner's permission, under independently verifiable conditions.
Passing either test would accomplish Multa Lumina's central research objective.
Why make this a collaborative project?
I don't know whether either challenge can be solved through the approach we're proposing.
I also don't know how many people it would take.
Perhaps an experienced researcher identifies a fundamental issue nobody else considered. Perhaps a beginner asks a question that exposes a flaw in an evaluation. Perhaps someone using an AI coding assistant develops a useful experimental tool.
That's part of what interests me.
What might happen if independent developers, working with different AI systems and approaches, openly shared their experiments, failures, improvements, and discoveries toward a common objective?
Rather than prescribe a particular architecture or model provider, I want Multa Lumina to remain open to evidence.
Here's one small example
Suppose an AI customer service employee encounters a change in company policy.
Previously, customers could request refunds within 30 days. The company has changed that period to 14 days.
However, an older internal document still says 30 days.
A customer now requests a refund after 20 days.
What should the AI do?
Should it trust the most recently updated document? Ask a coworker? Look for an official policy announcement? Recognize that the documents conflict before making a commitment?
And how would we determine whether its decision was correct?
This is the kind of scenario someone could help investigate without training a model or writing a complicated agent framework.
It also illustrates why the Full Job Test goes beyond conventional task completion.
You don't have to be an AI researcher
Multa Lumina is in its founding phase. The GitHub repository contains proposed evaluation protocols, contributor guidelines, research discussions, and initial opportunities.
People can contribute in several ways:
- Experienced developers: Build experimental agents, evaluation tools, and research infrastructure.
- AI-assisted developers and vibe coders: Investigate specific problems using their preferred development tools.
- Researchers: Challenge assumptions, propose methods, and help establish rigorous evidence standards.
- Beginners: Identify edge cases, improve documentation, ask useful questions, and contribute real-world scenarios.
- People with workplace or robotics experience: Help define what successful performance should actually look like.
We are also developing a public Discovery Ledger to document significant contributions, preserve research history, and recognize original insights, implementations, and independent verification. The detailed attribution procedures remain under review.
There is no required AI provider, predetermined number of contributors, or deadline.
What would count as meaningful progress?
I don't want to confuse impressive demonstrations with verified autonomy.
We should be able to distinguish a proposed hypothesis from an experimental result, and a promising result from an independently reproduced one.
If an approach fails, we document what happened and what we learned.
If an experiment identifies an important weakness in planning, memory, reliability, or physical reasoning, that's useful research too.
Multa Lumina isn't a claim that AGI has been achieved or that we're certain to achieve it.
It's an invitation to investigate two difficult problems openly.
Want to explore the project?
Even if you aren't ready to contribute code, you can help by questioning our assumptions, proposing difficult test scenarios, or identifying weaknesses in the evaluation methods.
The repository is public:
If you're interested but unsure where to begin, the Start Here guide explains how people at different experience levels can participate.
You can also browse the open research issues or join the community discussions.
I'll leave you with a question.
What would you need to observe before you felt confident that an AI agent could independently manage a real job, rather than simply complete a collection of tasks?
I'd genuinely welcome your thoughts, especially if you disagree with our proposed approach.
Multa Lumina. Many lights. One pursuit.
Top comments (0)