Skip to content

update documentation for regi-python #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
adamkorynta wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

update documentation for regi-python #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# AGENTS.md

## Repository Overview

`regi-headless` now uses a **Python-first execution model**.

- `regi-headless/src/main/python/regi_python` is the runtime bridge. It starts the JVM with **JPype** and exposes `regi_session()` and `run_headless(calculation_callback)`.
- `district-scripts/` and the example scripts under `regi-headless/src/test/resources/...` are now Python entrypoints that call into Java through the bridge.

Treat this repo as a Python orchestration layer over REGI Java libraries.

## Current Architecture

- `regi-headless/src/main/python/regi_python/regi_python.py` is the current Python entrypoint.
- `regi_session()` owns JVM startup/shutdown.
- `run_headless()` creates the REGI domain and invokes a Python callback with a Java-backed registry.
- `regi-headless/src/main/java/...` contains the Java support layer that the Python bridge calls through JPype.
- `district-scripts/` contains active district scripts written calling Java-backed calculations inside the callback.
- `regi-headless/src/test/resources/usace/rowcps/headless/examples` contains the same pattern for example scripts used by tests.

## File Groups To Treat Differently

- `regi-headless/src/main/python/`
- python to java bridge code
- prefer modern Python style and small, testable helpers
- `district-scripts/`
- active district-owned operational scripts
- preserve behavior unless the task explicitly changes calculation results or API usage
- `regi-headless/src/test/resources/usace/rowcps/headless/examples/`
- example scripts used by tests
- keep them aligned with the district-script pattern
- `docs/`
- useful for historical context and documentation of features moving forward
- `docs/agent-guides/` is the shared source of truth for agent-facing workflow guidance

## Working Rules

- Keep edits scoped to the requested migration target.
- When a script needs a new Java method, update the Java public API and the test harness together; the script test suite validates against the Java source-defined scriptable API.
- Avoid broad refactors outside the requested script family.

## Verification

Common checks in this repo:

- `./gradlew buildPythonWheel`
- `./gradlew testPythonWheel`

The Python package is built from `regi-headless/src/main/python`, and the wheel test verifies the package metadata, importability, public API exposure, and bundled Java jars.

## Environment Notes

The Python bridge expects the Java environment to be available, and the repo documentation currently references these variables:

- `JAVA_HOME`
- `CDA_URL`
- `CDA_API_KEY`
- `OFFICE_ID`
- `REGI_LOG_LEVEL`

Use the repo's current bridge code and tests as the source of truth for behavior; use the docs for orientation, not as a strict contract. `regi-headless/src/test/python/test_district_scripts.py` is the main contract for migrated script shape and allowed API calls.
112 changes: 71 additions & 41 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,75 +1,105 @@
# REGI-Headless

> [!IMPORTANT]
> **Notice: Project Refactor in Progress**
> This project is undergoing a large refactor for [CWMS Data API](https://github.com/USACE/cwms-data-api) support.
> It will transition from a Java project consuming Jython scripts to a **Python project** that
> utilizes **JPype** to call underlying REGI Java libraries.
`regi-python` is the Python bridge over the REGI Java libraries.

`REGI-Headless` is a Java-based command-line tool and library designed to run
**REGI** calculations in a headless environment.
It allows users to execute complex hydrological calculations and manage gate settings via Jython
scripts without the need for a graphical interface.
The client-facing Python API is documented in [docs/PYTHON_API.md](docs/PYTHON_API.md).
For script migration guidance, see [docs/JYTHON_TO_JPYPE_MIGRATION.md](docs/JYTHON_TO_JPYPE_MIGRATION.md).

## Features
## What Lives Where

- **Headless Execution**: Run REGI calculations as part of automated workflows or on servers.
- **Database Integration**: Connects to CWMS data retrieval and storage.
- **Modular Calculations**: Includes support for:
- Inflow calculations (Clone, Compute, Auto-Adjust, Balance All, etc.)
- Flow Group and gate settings calculations.
- `regi-headless/src/main/python/regi_python/`
- Python bridge code
- Public entry points: `regi_session()` and `run_headless(calculation_callback)`
- `regi-headless/src/main/java/`
- Java support layer used by the bridge calling into REGI calculation and data access libraries
- `district-scripts/`
- Copy of district-owned Python scripts used as examples for smoke tests

## Project Structure
## Requirements

- `regi-headless/`: Core Java implementation, including `RegiCLI`.
- `district-scripts/`: Example scripts and district-specific configurations.
- `docs/`: Additional documentation.
- Java JDK 21 or higher
- Python 3.11 or higher

## Getting Started
### Environment variables
- `JAVA_HOME` set for JPype startup
- `CDA_URL` url for CDA instance to connect to
- `CDA_API_KEY` required for accessing and storing data in CDA
- `OFFICE_ID` session scoped office for data access
- `REGI_LOG_LEVEL` for logging verbosity
- `REGI_LOG_FORMAT` overrides the default Python log format

### Prerequisites
## Building The Wheel

- Java JDK 21 or higher.
Build the Python wheel with Gradle:

### CWMS Data API Configuration
```powershell
./gradlew buildPythonWheel
```

The wheel is written to `regi-headless/build/install/regi_python/dist/`.

The library reads the following environment variables when establishing the CWMS Data API data source:
Release tags become wheel versions and must be PEP 440 compatible. For example, use `0.0.2a0`, `0.0.2b0`, or `0.0.2rc0` instead of `0.0.2-alpha`, `0.0.2-beta`, or `0.0.2-rc`.

- `CDA_URL`: Base URL for the CWMS Data API endpoint.
- `API_KEY`: API key used for CWMS Data API authentication and authorization.
- `OFFICE_ID`: CWMS office identifier used to scope the session.
Install the built wheel into a Python environment:

```powershell
pip install regi_python-*.whl
```

The factory uses these values to authenticate, resolve the current user, and persist the connected office and time zone into the REGI project.
### Notes

### Building
- The Python package name is `regi_python`.
- The wheel metadata name is `regi-python`.
- The bundled Java jars are packaged inside `regi_python/lib/`.

## Releases

Releases are published from the GitHub repository at [USACE-WaterManagement/regi-python](https://github.com/USACE-WaterManagement/regi-python). A release build attaches the Python wheel and checksum file to the GitHub Release for the matching tag.

To consume a published wheel, download the wheel asset from the release and install it with your package manager. Use the exact wheel filename from the release asset URL.

```powershell
./gradlew build
pip install https://github.com/USACE-WaterManagement/regi-python/releases/download/<tag>/regi_python-<version>-py3-none-any.whl
```

Details TBD.
```powershell
uv pip install https://github.com/USACE-WaterManagement/regi-python/releases/download/<tag>/regi_python-<version>-py3-none-any.whl
```

## Usage
```powershell
poetry add https://github.com/USACE-WaterManagement/regi-python/releases/download/<tag>/regi_python-<version>-py3-none-any.whl
```

```powershell
pdm add https://github.com/USACE-WaterManagement/regi-python/releases/download/<tag>/regi_python-<version>-py3-none-any.whl
```

If your tool does not support direct wheel URLs, download the asset from the release page and install it from the local `.whl` file instead.

TBD
## Using The Bridge

### Command Line Options
See [docs/PYTHON_API.md](docs/PYTHON_API.md) for the full Python-facing contract.

TBD
## Verification

### Example
Run the wheel smoke tests:

TBD
```powershell
./gradlew testPythonWheel
```

Run the script/API compatibility smoke test:

## Testing
```powershell
./gradlew smokeTestDistrictScripts
```

TBD
`./gradlew check` runs both along with Java unit tests.

## Maintainers

See [MAINTAINERS.md](MAINTAINERS.md) for a list of project maintainers.
See [MAINTAINERS.md](MAINTAINERS.md).

## License

See [LICENSE](LICENSE) for licensing information.
See [LICENSE](LICENSE).
122 changes: 122 additions & 0 deletions docs/JYTHON_TO_JPYPE_MIGRATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Migrating REGI Headless District Jython Scripts to `regi_python`

This guide captures the script migration pattern used in the district-scripts history,
moving from old Python 2 Jython-style top-level scripts to the current JPype-based `regi_python` bridge.

## Core Migration Pattern

Old scripts typically:

- imported Java classes at module import time
- accessed global `registry` and calculation objects at top level
- executed the calculation immediately when the file was loaded
- relied on shell wrappers such as `RunHeadlessJython.bat` or `.sh`

New scripts should:

- import `regi_session` and `run_headless` from `regi_python`
- move Java imports inside a callback that runs after the JVM starts
- wrap the calculation logic in a function such as `run_calculations(registry)`
- keep `registry.getCalculation(...)` and Java method calls unchanged
- end with `if __name__ == "__main__": with regi_session(): run_headless(run_calculations)`

## Before And After

### Before

```python
from java.util import Calendar
from java.util import TimeZone

gateCalc = registry.getCalculation(1.0, "Gate Flow")
timeZone = TimeZone.getTimeZone("US/Central")
startCal = Calendar.getInstance(timeZone)
endCal = Calendar.getInstance(timeZone)
gateCalc.computeFlowGroup("SWT", "FOSS", startCal.getTime(), endCal.getTime(), "Flow.FOSS.Project_Total")
```

### After

```python
from regi_python import regi_session, run_headless


def run_calculations(registry):
from java.util import Calendar
from java.util import TimeZone

gateCalc = registry.getCalculation(1.0, "Gate Flow")
timeZone = TimeZone.getTimeZone("US/Central")
startCal = Calendar.getInstance(timeZone)
endCal = Calendar.getInstance(timeZone)
gateCalc.computeFlowGroup("SWT", "FOSS", startCal.getTime(), endCal.getTime(), "Flow.FOSS.Project_Total")


if __name__ == "__main__":
with regi_session():
run_headless(run_calculations)
```

## Translation Rules

### 1. Move Java imports inside the callback

JPype-backed imports should happen after the JVM starts.
Put them inside `run_calculations(registry)` or a nested helper only called from that callback.

This is the biggest behavioral difference from Jython scripts.

### 2. Keep the registry lookup pattern

The migration does not change the scriptable REGI API. Calls like these stay the same:

- `registry.getNames(1.0)`
- `registry.getCalculation(1.0, "Inflow")`
- `registry.getCalculation(1.0, "Gate Flow")`

The change is only the Python wrapper around those calls.

### 3. Convert top-level execution into a callback

Scripts must be converted from immediate execution to a `run_calculations()` function.
That keeps import side effects out of the module.

### 4. Replace Python 2 `print` statements

Old scripts often contain lines like:

```python
print "Error Computing Flow Group"
```

Update these to Python 3 syntax:

```python
print("Error Computing Flow Group")
```

### 5. Keep optional Java-side logging calls

Calls such as `LoggingOptions.setDbMessageLevel(2)` and `LoggingOptions.setMetricsEnabled(True)` still belong in the script if the district workflow depends on them. They do not move to `regi_python`; they simply live inside the callback now.

### 6. Remove the shell wrapper

The old `RunHeadlessJython` launchers are no longer needed. The Python file itself becomes the entry point:

```python
if __name__ == "__main__":
with regi_session():
run_headless(run_calculations)
```

## Project-Specific Examples

The current district scripts show the same migration shape across several script families:

- `SWF/InflowCalcComputedInflow.py`
- `SWF/InflowCalcComputeEvapAsFlow.py`
- `SWF/GateSettings.py`
- `SWT/GateFlowGroup1.py`
- `SWL/Big3-GateFlow.py`

The differences between them are the calculation names, location lists, and optional logging or flow-group loops. The migration pattern itself is the same.
Loading
Loading