diff --git a/README.md b/README.md index 89baae7..80a235d 100644 --- a/README.md +++ b/README.md @@ -116,3 +116,13 @@ python .\scripts\run_release_smoke.py --root . --mode live-public Для локального воспроизводимого прогона есть отдельный runbook: `docs/runbooks/reproducible_local_runbook.md`. Для отдельного тяжелого smoke-прогона есть workflow `.github/workflows/release_smoke.yml`. Для честной сверки реализации со спецификацией есть `docs/status/spec_coverage_map.yaml` и `docs/status/spec_gap_audit.md`. + +## Developer docs + +- `docs/development/onboarding.md` — быстрый путь от clone до reviewable local run. +- `docs/development/local_setup.md` — установка и editable/dev режим. +- `docs/development/commands.md` — Makefile, CLI и quality gate команды. +- `docs/development/environment.md` — безопасная работа с env и adapter variables. +- `docs/development/troubleshooting.md` — типовые локальные сбои и диагностика. +- `docs/development/project_structure.md` — ответственность ключевых папок. +- `docs/development/contribution_workflow.md` — branch, commit и PR workflow. diff --git a/docs/development/commands.md b/docs/development/commands.md new file mode 100644 index 0000000..1ab8253 --- /dev/null +++ b/docs/development/commands.md @@ -0,0 +1,36 @@ +# Command reference + +Use this page as the local command index. The Makefile exposes the shortest +aliases, while the Python module commands are useful when debugging from a shell +or CI step. + +## Make targets + +| Command | Purpose | +| --- | --- | +| `make bootstrap` | Create local bootstrap artifacts through the CLI. | +| `make validate-config` | Validate the default configuration contract. | +| `make dry-run` | Execute the pipeline dispatcher without material side effects. | +| `make test` | Run the default pytest suite. | + +## Python commands + +| Command | Purpose | +| --- | --- | +| `python -m alpha_research config-validate` | Validate config files and adapter contracts. | +| `python -m alpha_research bootstrap` | Prepare local runtime directories and manifests. | +| `python -m alpha_research run-full-pipeline --dry-run` | Check the orchestration graph without running the full research workload. | +| `python .\scripts\verify_release_bundle.py --root .` | Verify release evidence artifacts. | +| `python .\scripts\run_release_smoke.py --root . --mode configured-local` | Run deterministic configured-local smoke. | +| `python .\scripts\run_release_smoke.py --root . --mode live-public` | Run live-public smoke against external providers. | + +## Quality gates + +```bash +python -m ruff check src tests +python -m mypy src/alpha_research +python -m pytest +``` + +Run the smaller unit/integration slices while developing. Run the full set before +opening a pull request that touches shared pipeline behavior. diff --git a/docs/development/contribution_workflow.md b/docs/development/contribution_workflow.md new file mode 100644 index 0000000..5c7b39c --- /dev/null +++ b/docs/development/contribution_workflow.md @@ -0,0 +1,41 @@ +# Contribution workflow + +Use small branches and keep every commit tied to one reviewable change. + +## Branch checklist + +1. Start from `master`. +2. Create a branch with a scope prefix: `docs/`, `fix/`, `test/`, `refactor/`, + `ci/`, or `chore/`. +3. Make the smallest coherent change. +4. Add or update tests when behavior changes. +5. Run the local checks that match the touched area. +6. Open a pull request with validation notes and linked issue. + +## Commit style + +Use Conventional Commits: + +- `docs: add reproducible run notes` +- `test: cover configured adapter fallback` +- `fix: reject same-bar execution timestamps` +- `refactor: isolate report bundle writer` +- `ci: add release bundle verification step` + +Avoid vague messages such as `update`, `fix`, `changes`, or `wip`. + +## Pull request validation + +At minimum, record: + +- install command or dependency state; +- lint/typecheck result when Python code changed; +- pytest target(s) run; +- release smoke or reason it was not relevant; +- manual review notes for docs-only changes. + +## Artifact discipline + +Do not commit local `.env`, caches, generated dependency directories, ad hoc +notebooks, or release outputs unless the repository explicitly treats the +artifact as source-controlled evidence. diff --git a/docs/development/environment.md b/docs/development/environment.md new file mode 100644 index 0000000..41ea30c --- /dev/null +++ b/docs/development/environment.md @@ -0,0 +1,32 @@ +# Environment variables + +The repository should run in deterministic local mode without private secrets. +Live provider credentials are optional and should stay in a local `.env` file or +the shell environment. + +## Core variables + +| Variable | Required | Example | Notes | +| --- | --- | --- | --- | +| `ALPHA_RESEARCH_ENV` | no | `local` | Describes the active runtime environment. | +| `ALPHA_RESEARCH_TRACKING_URI` | no | `./mlruns` | Local tracking output directory. | +| `ALPHA_RESEARCH_LOG_LEVEL` | no | `INFO` | Logging verbosity for local runs. | + +## Configured adapter variables + +Configured adapters may reference variables through `api_key_env`, +`user_agent_env`, or `local_path_env` in YAML config. Keep those values out of +Git unless they are harmless placeholders. + +| Variable pattern | Purpose | Safe example | +| --- | --- | --- | +| `*_API_KEY` | API key for a live provider. | `replace-me` | +| `*_USER_AGENT` | Provider-specific user agent. | `alpha-research-local/0.1` | +| `*_LOCAL_PATH` | Local CSV, JSON, or Parquet fixture path. | `./tests/fixtures/provider.csv` | + +## Secret handling rules + +- commit `.env.example`, never a real `.env`; +- use fake placeholder values in documentation; +- prefer configured-local smoke for pull requests; +- run live-public smoke only when external access is intended and documented. diff --git a/docs/development/local_setup.md b/docs/development/local_setup.md new file mode 100644 index 0000000..48f8caf --- /dev/null +++ b/docs/development/local_setup.md @@ -0,0 +1,47 @@ +# Local setup + +The project targets Python 3.12 and keeps runtime and developer dependencies in +lock files. Prefer a dedicated virtual environment so release smoke artifacts and +cache files do not mix with other projects. + +## Windows PowerShell + +```powershell +py -3.12 -m venv .venv +.\.venv\Scripts\Activate.ps1 +python -m pip install --upgrade pip +python -m pip install -r requirements.lock -r requirements-dev.lock +``` + +## macOS and Linux + +```bash +python3.12 -m venv .venv +source .venv/bin/activate +python -m pip install --upgrade pip +python -m pip install -r requirements.lock -r requirements-dev.lock +``` + +## Editable install + +Use an editable install when changing package code or CLI entry points: + +```bash +python -m pip install -e ".[dev,research]" +``` + +The lock-file install is better for release reproduction. The editable install is +better for local feature work. + +## Smoke check + +After installation, run: + +```bash +python -m alpha_research config-validate +python -m alpha_research run-full-pipeline --dry-run +python -m pytest tests/unit +``` + +These commands should complete without network access. Network-backed adapter +checks belong in explicit live-public smoke runs. diff --git a/docs/development/onboarding.md b/docs/development/onboarding.md new file mode 100644 index 0000000..3ef2f3c --- /dev/null +++ b/docs/development/onboarding.md @@ -0,0 +1,44 @@ +# Developer onboarding + +This guide is the short path from a fresh clone to a reviewable local run of the +Alpha Research Platform. The canonical product and research contracts still live +in `docs/specs/MASTER_SPEC.md`, but this document explains the engineering +sequence a contributor should follow before opening a pull request. + +## First read + +Read these files in order: + +1. `README.md` for the platform intent and current capability map. +2. `docs/specs/MASTER_SPEC.md` for the research and operational invariants. +3. `docs/specs/machine_spec.yaml` for machine-readable stage contracts. +4. `docs/runbooks/reproducible_local_runbook.md` for deterministic local runs. +5. `docs/status/spec_coverage_map.yaml` for the implementation-to-spec map. + +## Local confidence path + +The lightweight local confidence path is: + +```bash +python -m pip install -r requirements.lock -r requirements-dev.lock +python -m alpha_research config-validate +python -m pytest tests/unit +python -m pytest tests/integration/test_config_loader.py +python .\scripts\run_release_smoke.py --root . --mode configured-local +``` + +Use the full CI-equivalent path before merging changes that affect the pipeline, +release bundle, point-in-time joins, OOF training, or reporting outputs. + +## Review posture + +Every change should preserve these invariants: + +- no same-bar execution in backtests; +- no future fundamentals in point-in-time joins; +- preprocessing is fit only on train folds; +- release-grade runs do not silently fall back to fixture-only data; +- reports identify uncertainty, stability, and FDR evidence. + +If a change intentionally relaxes one of these rules, document the reason in the +pull request and update the relevant spec/status artifact in the same branch. diff --git a/docs/development/project_structure.md b/docs/development/project_structure.md new file mode 100644 index 0000000..5e92834 --- /dev/null +++ b/docs/development/project_structure.md @@ -0,0 +1,29 @@ +# Project structure + +The repository is organized around research validity first and implementation +convenience second. Keep new code close to the invariant it protects. + +| Path | Responsibility | +| --- | --- | +| `configs/` | Runtime configuration and adapter contracts. | +| `schemas/` | Machine-readable data and feature registry contracts. | +| `src/alpha_research/config` | Config loading, validation, hashing, and snapshots. | +| `src/alpha_research/time` | Trading calendars and decision/execution timestamp rules. | +| `src/alpha_research/data` | Provider contracts, raw/bronze ingest, QA, and storage. | +| `src/alpha_research/pit` | Point-in-time interval logic and as-of joins. | +| `src/alpha_research/features` | Registry-driven feature engineering. | +| `src/alpha_research/labels` | Label generation with explicit forward windows. | +| `src/alpha_research/splits` | Purged walk-forward split construction. | +| `src/alpha_research/models` | Baseline models and model-facing interfaces. | +| `src/alpha_research/training` | OOF training and tuning orchestration. | +| `src/alpha_research/portfolio` | Target weights, constraints, and turnover handling. | +| `src/alpha_research/execution` | Execution simulation and cost handling. | +| `src/alpha_research/evaluation` | Metrics, uncertainty, stability, and reports. | +| `src/alpha_research/pipeline` | Stage graph, dispatcher, and runtime handlers. | +| `tests/unit` | Fast deterministic checks for pure logic. | +| `tests/integration` | Cross-module pipeline and artifact checks. | +| `tests/leakage` | Time and fold leakage guards. | +| `docs/status` | Machine-readable implementation and acceptance evidence. | + +When adding a new subsystem, add tests in the same branch and update the status +map when the change enforces or documents a specification clause. diff --git a/docs/development/troubleshooting.md b/docs/development/troubleshooting.md new file mode 100644 index 0000000..6b3635e --- /dev/null +++ b/docs/development/troubleshooting.md @@ -0,0 +1,47 @@ +# Troubleshooting + +This page lists common local failures and the shortest useful recovery path. + +## Import errors after cloning + +Install the package in editable mode: + +```bash +python -m pip install -e ".[dev,research]" +``` + +If the error only appears in an IDE, make sure the IDE interpreter points to the +same virtual environment used by the shell. + +## Config validation fails + +Run: + +```bash +python -m alpha_research config-validate +``` + +Then inspect the failing config path and field name. Unknown fields are treated +as contract drift, not harmless extras. + +## Configured adapter cannot find local data + +Check whether the adapter uses `local_path` or `local_path_env`. Relative paths +are resolved from the repository root during local runs. + +## Release smoke fails in live-public mode + +Live-public mode can fail because of provider availability, rate limits, or +missing external configuration. Reproduce with configured-local mode first: + +```bash +python .\scripts\run_release_smoke.py --root . --mode configured-local +``` + +Only debug external transport after the configured-local path is clean. + +## Leakage tests fail + +Treat leakage failures as correctness failures. Re-run the targeted test, then +inspect timestamp joins, fold boundaries, preprocessing fit points, and any +shortcut that may have pulled future data into the decision timestamp.