Back to articles
System Design

Building Scalable Online Judge Systems

A deep dive into creating competitive programming platforms with load balancing, security, and multi-language support. Lessons learned from building Andalus Judge for Ethiopian programmers.

January 6, 202612 min read

Introduction

Building an online judge system is one of the most challenging and rewarding projects I've undertaken. When I started Andalus Judge, my goal was simple: create a platform where Ethiopian programmers could practice competitive programming without relying on international platforms with high latency.

The Architecture

An online judge has several critical components:

1. Submission Queue

We used Celery with Redis as the message broker to handle the queue of submissions. This allows us to process submissions asynchronously and scale horizontally.

2. Sandboxed Execution

Security is paramount. Each submission runs in an isolated Docker container with:

  • Limited CPU time
  • Memory restrictions
  • No network access
  • Read-only filesystem (except for specific directories)

    • 3. Test Case Management

    Test cases are stored separately and streamed to the judge during evaluation. This prevents memory issues with large test files.

    Challenges Faced

    Load Balancing

    During contests with 200+ participants, we needed to distribute the load across multiple judge servers. We implemented a custom load balancer that considers:

  • Current queue depth per server
  • Server resource utilization
  • Problem difficulty (some problems need more resources)

    • Security Concerns

    Users will try to break your system. We encountered:

  • Fork bombs
  • Infinite loops
  • Attempts to read other users' solutions
  • Memory allocation attacks

    • Each required specific countermeasures in our sandbox configuration.

    Results

    After two years of development and iteration:

  • Hosted 50+ contests
  • 5000+ registered users
  • Processing 10,000+ submissions monthly
  • 99.9% uptime during contests

    • Lessons Learned

  • **Start with security** - It's much harder to add security later
  • **Monitor everything** - You can't fix what you can't measure
  • **Plan for scale** - Even if you don't need it immediately
  • **Community feedback** - Users will find bugs you never imagined

    • The journey of building Andalus Judge taught me more about system design than any course or book could.