Skip to content

Elios-Lab/epic

Repository files navigation

EPIC - ELIOS Predictive Intelligence Challenge

EPIC is a competition platform for predictive intelligence on live digital twins. It turns a simulated physical system into a real-time machine learning challenge: participants connect to a sensor stream, collect data, predict a hidden future window, and are scored automatically, all without ever seeing the ground truth.

Resource URL
Live platform https://epic.elioslab.net
REST API https://epic.elioslab.net/api/v1
API live docs https://epic.elioslab.net/docs
SDK pip install epic-elios-client

Concept

Most machine learning competitions start with a file. EPIC starts with a system.

That system is a digital twin: a compact simulation of a physical asset, a mass-spring-damper, a centrifugal pump, an electric motor, a gearbox, a smart building, or any other physical system. The twin evolves in real time on the server following its internal physics (Newton's laws, fluid dynamics, thermodynamics, etc.). Faults can be scheduled inside the twin and alter the latent physics: the spring gets weaker, the pump cavitates, the motor overheats. Sensors observe the twin's internal state and produce noisy, biased, drifting, delayed, quantized, saturated, and sometimes false or outlying measurements.

Participants only receive the sensor stream. They never see the clean state, fault labels, or future observations. The platform stores those private signals and closes the stream before the evaluation window begins. Participants must forecast the future from what they have observed.

The result is a richer contest format than a static benchmark. Students and researchers practice the whole predictive-intelligence loop: instrumentation, data collection, temporal reasoning, modelling, submission integrity, and live leaderboard feedback.


Architecture

EPIC separates competition infrastructure from simulated domains:

EPIC architecture

At runtime, EPIC is a layered RESTful API application. The API layer handles authentication, contest management, registrations, submissions, leaderboards, invitations, and streams. The database layer stores users, contests, tasks, sessions, observations, submissions, scores, and leaderboard entries. The plugin registries keep digital twins, sensors, scoring metrics, and evaluators decoupled from the platform core, so new simulated systems can be added without rewriting the API. When a contest becomes active, the simulation engine instantiates the selected twin and sensors, runs the live session, streams observations to registered participants, stores hidden evaluation data, and triggers scoring after submissions. The web interface and the client SDK both sit on top of the same REST and WebSocket contract.


Contests

An EPIC contest moves through a lifecycle:

DRAFT -> SCHEDULED -> ACTIVE -> CLOSED -> ARCHIVED
  |                       |
  +-------> ACTIVE        v
                        PAUSED -> ACTIVE

The active phase is split into three time windows:

Window Time Range What Happens
Observation start_date to end_of_observation The simulation runs and registered participants receive sensor readings over WebSocket.
Evaluation end_of_observation for prediction_horizon_seconds The simulation continues, the stream is closed, and private ground truth is recorded.
Submission after evaluation until end_date Participants submit forecasts for the hidden evaluation window.

Creating a Contest

Contest creation is fully configuration-driven. Organizers choose the simulated system, the sensors participants will see, the target variables they must predict, the initial conditions and optional fault schedule, the scoring metric, and the contest timeline. Several templates are available to quickly launch contests with pre-configured twins, sensors, and tasks. A representative request:

{
  "name": "Pump Bearing Wear Challenge",
  "description": "Forecast flow and vibration during progressive bearing wear.",
  "visibility": "PUBLIC",
  "task_type": "FORECASTING",
  "metric_ids": ["mae"],
  "twin_id": "industrial_pump",
  "fault_schedule": [
    {
      "fault_id": "bearing_wear",
      "start_time": 20.0,
      "end_time": null,
      "severity": 0.7
    }
  ],
  "sensor_configs": [
    {"sensor_id": "flow_rate", "noise_std": 0.2},
    {"sensor_id": "pressure", "noise_std": 0.02},
    {"sensor_id": "temperature", "noise_std": 0.05},
    {"sensor_id": "vibration", "noise_std": 0.03}
  ],
  "target_variables": ["flow_rate", "vibration"],
  "initial_conditions": {
    "flow_rate": 120.0,
    "pressure": 4.0,
    "wear": 0.05
  },
  "sampling_rate_hz": 10.0,
  "score_against": "ground_truth",
  "start_date": "2027-01-10T09:00:00Z",
  "end_of_observation": "2027-01-10T09:30:00Z",
  "prediction_horizon_seconds": 60.0,
  "end_date": "2027-01-10T09:40:00Z"
}

Forecasting Task

The implemented task evaluator is FORECASTING. EPIC computes the number of steps participants must predict:

eval_steps = round(prediction_horizon_seconds * sampling_rate_hz)

Target variables are a non-empty subset of configured sensors. Other sensors can be streamed as explanatory features but do not affect the score. Each submission must contain one list of exactly eval_steps values per target variable:

{
  "forecast": {
    "position": [0.12, 0.13, 0.14],
    "velocity": [1.8, 1.7, 1.6]
  }
}

Validation rules:

  • every configured target variable must be present;
  • each list must contain exactly eval_steps values;
  • values must be numeric;
  • extra forecast keys are accepted but ignored for scoring.

Task configuration fields:

Field Meaning
eval_steps Number of predicted values per target variable.
target_variables Configured sensor ids required and scored.
score_against ground_truth or sensors.
metric_ids Registered metrics to compute.

ground_truth compares against clean latent values recorded before sensor corruption. sensors compares against noisy sensor readings when the contest is about predicting the measured signal itself.

Built-in metrics:

Metric Direction Purpose
mae minimize Mean absolute error for forecasting.
f1 maximize Binary F1 score for anomaly-detection style tasks.

Leaderboards keep each participant's best evaluated submission, respecting the metric direction.


Roles

EPIC has three roles:

Role Scope
PARTICIPANT Join contests, stream data, submit forecasts, get scores.
ORGANIZER Create and manage contests, invite participants, inspect submissions, pause and resume sessions.
ADMINISTRATOR Manage users, organizer requests, all contests, and platform operations.

Access is intentionally controlled. Participants cannot self-register or apply to join contests — they are always admitted through an organizer or an administrator. This workflow is designed for educational and research environments where organizers are typically teachers or researchers managing contests for a predefined group.

Organizer

Prospective organizers submit a registration request, which an administrator reviews and approves or rejects. Once approved, they can create a contest from a template or define one from scratch (see Creating a Contest). Once configured, a contest can be scheduled or activated immediately. During the contest, organizers can invite participants, monitor registrations and submissions, inspect the leaderboard, extend deadlines, and pause, resume, or close the session.

Participant

The maintained participant walkthrough is the quickstart notebook:

notebooks/quickstart.ipynb

It covers the complete workflow: installing the SDK, logging in, finding an active contest, registering, collecting live sensor data, building a forecast, submitting it, and reading scores and leaderboard results. It can be run locally with Jupyter or opened directly in Colab using the badge at the top of the notebook.

Administrator

Administrators keep the platform usable and accountable. They review organizer access requests, create accounts when needed, promote users to organizer or administrator roles, and suspend or restore access. They have full contest oversight — they can inspect every contest regardless of owner, intervene in lifecycle operations, and impersonate active users to reproduce workflow problems. To simplify initial deployment, a bootstrap administrator account can be created at startup using the ADMIN_USERNAME and ADMIN_PASSWORD environment variables.


API

The endpoint-level contract is generated by FastAPI at /docs. All protected REST endpoints use:

Authorization: Bearer <JWT>

Core route groups:

Area Routes
Auth POST /api/v1/auth/login, GET /api/v1/auth/me
Users POST/GET /api/v1/users, GET/PATCH/DELETE /api/v1/users/{user_id}, impersonation
Organizer requests public request creation, admin list/approve/reject
Contests create, list, read, update status/deadline, pause, resume, delete
Invitations create/list/revoke contest invitations, validate/accept public token
Registrations join, list, inspect, withdraw/remove
Streaming WS /api/v1/ws/contests/{contest_id}?token=...
Submissions create/list/read submissions and scores
Leaderboards public contest leaderboard and permission-checked user entry
Metadata templates, catalog, twin metadata, compatible sensors, faults

Business-rule failures use a stable error envelope:

{
  "error": {
    "code": "CONTEST_STATE_ERROR",
    "message": "Contest is not active"
  }
}

Common error codes include INVALID_CREDENTIALS, FORBIDDEN, CONTEST_NOT_FOUND, CONTEST_STATE_ERROR, REGISTRATION_ERROR, SUBMISSION_ERROR, VALIDATION_ERROR, PLUGIN_NOT_FOUND, and PLUGIN_EXECUTION_ERROR.

WebSocket Stream

Participants connect with the token in the query string:

wss://<host>/api/v1/ws/contests/{contest_id}?token=<JWT>

Each observation message carries a sensor snapshot:

{
  "timestamp": "2027-01-15T10:00:00.500000+00:00",
  "session_id": "8d44f402-0000-0000-0000-000000000000",
  "sequence_id": 116,
  "committed_through": 110,
  "sensors": {
    "position": 0.15,
    "velocity": 1.82
  }
}

sequence_id increments every simulation step. committed_through is the highest sequence safely flushed to the database. The stream never includes ground truth, labels, or the twin's internal state.

When the observation phase ends, the server signals the transition and closes the stream:

{
  "event": "evaluation_started",
  "observation_end_sequence_id": 400,
  "evaluation_steps": 20
}

If a contest is closed early, participants receive:

{ "event": "contest_closed" }

Local Development

Requirements:

  • Python 3.11 or later
  • uv
  • SQLite for local development, PostgreSQL for production

Install:

git clone https://github.com/Elios-Lab/epic.git
cd epic
uv sync

Create .env:

DATABASE_URL=sqlite+aiosqlite:///./epic.db
SECRET_KEY=change-me-in-production
ADMIN_USERNAME=admin
ADMIN_EMAIL=admin@example.com
ADMIN_PASSWORD=change-me
BASE_URL=http://localhost:8000

Run migrations:

set -a
source .env
set +a
uv run alembic upgrade head

Build the web interface before using FastAPI as the frontend host:

cd epic_core/gui
npm ci
npm run build
cd ../..

Start the server:

uv run uvicorn "epic_core.api.main:create_app" --factory --reload

For frontend development, run the API and Vite dev server separately. Vite proxies /api and WebSocket traffic to FastAPI on port 8000:

uv run uvicorn "epic_core.api.main:create_app" --factory --reload

cd epic_core/gui
npm ci
npm run dev

Open:

Useful optional settings:

Variable Default Purpose
ACCESS_TOKEN_EXPIRE_MINUTES 60 JWT lifetime
MAX_CONCURRENT_SESSIONS 50 Maximum active simulation runners in this API process
SESSION_QUEUE_CAPACITY 1000 Per-client WebSocket queue
BASE_URL http://localhost:8000 Invitation link base URL
SMTP_HOST unset Enables email notifications when configured
SMTP_PORT 587 SMTP port
SMTP_TLS true STARTTLS

Testing

Default suite:

uv run pytest tests/ --tb=short -q

Focused suites:

uv run pytest tests/core
uv run pytest tests/api
uv run pytest tests/twins
uv run pytest tests/sensors
uv run pytest epic_client/tests

The Playwright UI suite is excluded from the default run:

uv run pytest tests/ui

API tests use a per-test SQLite database, fresh plugin registries, FastAPI TestClient, and a collecting notification service. They must not use production settings or production registries.


Extending EPIC

See epic_plugins/twins/README.md for the extension guide, including how to add digital twins, sensors, metrics, and task evaluators.

Roadmap

Implemented:

  • domain-independent interfaces and plugin registries;
  • FastAPI backend with JWT authentication;
  • organizer requests, participant invitations, admin user management, and impersonation;
  • two-phase forecasting contests with WebSocket observation streams;
  • pause, resume, close, restart recovery, and session isolation;
  • configurable sensors and fault schedules;
  • target-variable forecasting with automatic scoring and leaderboards;
  • contest templates and twin catalog API;
  • static role-based GUI;
  • PyPI-ready participant SDK.

Planned:

  • anomaly detection, fault classification, and remaining-useful-life task evaluators;
  • runtime plugin governance;
  • larger-scale distributed simulation;
  • more digital twin domain packs.

Credits

EPIC is developed by Elios Lab at the University of Genoa.

The long-term goal is simple: a new competition should be configuration, not backend code; and a new application domain should be a plugin, not a rewrite.

About

ELIOS Predictive Intelligence Challenge (EPIC) is a simulation-driven machine learning competition platform developed by the ELIOS Laboratory

Resources

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors