For years, IFSul – Campus Charqueadas, a federal institute in southern Brazil, has hosted programming marathons for both high school and undergraduate students.
Like many educational institutions, our goal has always been straightforward: create opportunities for students to develop problem-solving skills, learn programming, and experience the challenge and excitement of competitive programming.
What most participants never see, however, is everything happening behind the scenes.
This is the story of how our contests evolved from manual grading to automated judging, why I decided to build a new platform instead of maintaining a legacy one, and how that project eventually became both an open-source judge system and an online competitive programming platform.
The Era of Manual Grading
When our programming marathons began, there was no automated judging system.
Teams would solve problems, submit their source code, and wait for organizers to evaluate the solutions manually.
The process was exactly as tedious as it sounds.
An organizer would collect the source file, compile it, execute it against test cases, compare the results with the expected output, and then determine whether the solution should be accepted.
For small competitions, this approach was manageable.
As participation grew, it quickly became clear that it would not scale.
If we wanted our competitions to continue growing, we needed automation.
Enter BOCA
To solve this problem, a colleague introduced BOCA into our competitions.
For those unfamiliar with it, BOCA is a contest management and judging system widely used in Brazil, including in competitions associated with the Brazilian Computer Society (SBC). It has been around for many years and has played an important role in the Brazilian competitive programming ecosystem.
For us, it represented a major improvement.
Instead of managing teams, submissions, and scoreboards manually, we now had a dedicated platform built specifically for programming contests.
The administrative side became significantly easier.
The judging side, however, was a different story.
While BOCA supports automatic judging, configuring and maintaining that infrastructure was never particularly straightforward. The platform reflects design decisions from a different era of software development, and getting everything configured correctly required navigating a considerable amount of documentation, scripts, permissions, and server-specific configuration.
In practice, automatic judging frequently failed at exactly the wrong moment: during the contest itself.
As a result, we often used BOCA primarily as a contest management tool while much of the actual judging continued to happen manually.
It worked.
But it wasn't the experience we wanted.
The Moment Everything Changed
Eventually, responsibility for running the programming marathon landed on my desk.
There was no crisis.
No catastrophic outage.
No dramatic handover.
The contest was still running, and BOCA was still there.
The problem was much simpler.
I knew almost nothing about administering BOCA.
So I did what any developer would do: I started reading documentation, installation guides, configuration files, and forum discussions.
The deeper I went, the less interested I became in becoming a BOCA expert.
To be clear, BOCA has served the competitive programming community well for decades.
But I had little desire to spend my time learning the intricacies of a large legacy system with a configuration process that felt unnecessarily complicated and difficult to maintain.
At some point, I stopped asking myself:
"How do I configure BOCA correctly?"
And started asking:
"If I were building a contest platform today, what would it look like?"
That question eventually became AutoJudge.
Starting From Scratch
When I decided to build AutoJudge, my objective wasn't simply to replace BOCA.
I wanted to solve the problems I personally experienced while preparing competitions.
Before writing a single line of code, I established a few principles:
- Deployment should be simple and reproducible.
- Contest setup should not require extensive system administration knowledge.
- The interface should be friendly for both organizers and competitors.
- The judging infrastructure should be reliable enough that organizers can trust it during live events.
- Educational institutions should be able to self-host the platform without fighting the infrastructure.
These principles guided every decision that followed.
AutoJudge began as a side project aimed at improving our local competitions.
What started as a tool built for a specific need gradually evolved into an open-source project that other institutions can also use and adapt to their own contests.
Building the Judge
At its core, AutoJudge consists of three major components:
- A web application used by participants and administrators.
- An API responsible for contest management.
- A judging system responsible for evaluating submissions.
One of the most important parts of any judge system is executing untrusted code safely.
Every submission received by the platform is code written by a participant. By definition, the system must execute programs that it did not create and cannot fully trust.
To address this, AutoJudge executes submissions inside isolated containerized environments.
When a solution is submitted:
- The submission enters the judging queue.
- The source code is prepared for execution.
- An isolated execution environment is created.
- Resource limits are applied.
- The solution is compiled and executed.
- The output is compared against the expected results.
- The execution environment is discarded.
Containerization is not a guarantee of perfect security, nor should it be considered one.
Its purpose is to provide a practical isolation boundary that reduces risk while remaining manageable for educational institutions.
For our use case, it offered a good balance between safety, reliability, and operational simplicity.
Adopting AutoJudge
After enough development and testing, we decided as a group to adopt AutoJudge in our programming marathons.
The difference was immediate.
Instead of worrying about fragile contest infrastructure, we could focus on creating better competitions.
Creating and managing problems became considerably easier thanks to the web-based interface.
The judging process became more reliable.
The interface was significantly more approachable for students.
Most importantly, organizers could spend less time fighting the platform and more time running the event.
That was the entire point.
Why I Open-Sourced It
As we continued using AutoJudge, I started thinking about a simple question.
If we had struggled with these problems, how many other schools and universities were facing the same situation?
Many educational institutions want to host programming contests.
Many instructors want to introduce competitive programming to their students.
Many communities want to organize local events.
What they often lack is the time and infrastructure expertise required to maintain complex systems.
That realization is what motivated me to open-source the project.
Today, the judge, API, and web platform are all available publicly.
My hope is that other institutions can benefit from the same tools we built for ourselves.
Repository:
Beyond Self-Hosting
The open-source platform solved our institutional problem.
But another idea kept coming back.
What about everyone who doesn't want to host anything?
What about students who simply want to practice?
What about teachers who want to create contests without provisioning servers, configuring databases, managing updates, or maintaining infrastructure?
Those questions eventually led to autojudge.io.
Introducing autojudge.io
autojudge.io builds on the same ideas behind AutoJudge while removing the need for self-hosted infrastructure.
Instead of downloading, deploying, configuring, and maintaining the platform yourself, you can simply create an account and start using it.
Students can:
- Solve public problems.
- Track their progress.
- Compare themselves against other competitors.
- Participate in contests hosted by institutions and communities.
Educators and organizers can:
- Create contests.
- Manage participants.
- Publish problem sets.
- Run competitions without worrying about servers or deployment.
One feature that quickly became central to the platform is the integrated web-based code editor.
Participants can read a problem statement, write code directly in the browser, execute their solution, test it against custom inputs, and submit it to the official judging queue without ever leaving the platform.
The editor provides a complete workflow for competitive programming:
Read the problem statement.
Write code.
Run and debug solutions.
Test against sample or custom cases.
Submit to the judge.
Review verdicts and execution results.
This makes the platform particularly accessible for newcomers, who can start solving problems immediately without installing compilers, configuring IDEs, or setting up local development environments.
For experienced competitors, it provides a convenient environment for quick practice sessions and online contests.
The goal is not to replace traditional development tools.
The goal is to reduce friction and make participation as straightforward as possible.
Some institutions prefer complete control and self-hosting through the open-source AutoJudge platform.
Others prefer a zero-configuration solution that is ready to use immediately.
I wanted to support both.
What's Next?
AutoJudge started because I didn't want to spend my time learning how to maintain a legacy contest platform.
I wanted something simpler.
Something easier to deploy.
Something easier to understand.
Something designed around the needs of students and educators rather than the constraints of decades-old infrastructure.
What began as a practical solution for a local programming marathon eventually evolved into an open-source project and an online platform used beyond our campus.
There is still a lot to improve.
The judging infrastructure continues to evolve.
New features continue to be developed.
And there are countless ideas still waiting to be implemented.
But the core mission remains unchanged:
Make programming contests easier to organize and more accessible to everyone.
If you're involved in programming education, competitive programming, or educational technology, I'd love to hear your experiences.
How does your institution handle programming contests?
What tools are you using today?
And have you ever looked at a legacy system and thought:
"I could probably build this myself."
Because that's exactly how AutoJudge started.
Top comments (0)