diff --git a/docs/quality-switch-spec.md b/docs/quality-switch-spec.md new file mode 100644 index 0000000..80ce1ee --- /dev/null +++ b/docs/quality-switch-spec.md @@ -0,0 +1,416 @@ +# Auto Quality Profile Switch — Specification + +## 1. Problem Statement + +Radarr profiles can pin movies to Remux-only quality. As studios increasingly skip physical (disc) releases, some movies never get a Remux source. Those movies stall permanently — never upgraded, never downloaded. The library accumulates "dead" entries that will never be satisfied. + +**Goal:** Automatically detect movies unlikely to ever get a physical release and switch them to a quality profile that allows WebDL. + +## 2. Research Summary + +Empirical analysis of 2,631 movies with cinema dates yields: + +### 2.1 Physical release timing + +| Metric | Value | Meaning | +|--------|-------|---------| +| P50 (median) | 54 days | Half of physical releases within 54 days of web | +| P90 | 195 days | 90% within ~6.5 months | +| P95 | 265 days | 95% within ~9 months | +| P99 | 545 days | 99% within ~18 months | +| Sample | 1,921 movies | Movies with both web + physical dates | +| Outliers removed | 407 | Bogus dates filtered via IQR | + +**Interpretation:** If a movie has no physical release 265 days after its web/streaming date, there is a ~5% chance a physical release will still come later. This is the recommended threshold. + +### 2.2 Current backlog + +Movies with cinema + web but no physical: **334** +Of those, **~290** have waited longer than 265 days. + +These are immediate candidates for profile switching. + +### 2.3 Assumptions and caveats + +- Data comes from TMDB via Radarr's metadata. TMDB may have incomplete or incorrect dates. +- `fromdateiso8601` in jq parses ISO 8601 date strings. Bogus epoch-zero dates or year-1900 values produce extreme outliers; IQR filtering catches most of these. +- The P95 threshold recomputes from your library on each run, adapting to your collection's distribution. +- Some legitimate physical releases genuinely take longer than 265 days (e.g., boutique labels, limited editions) — the 5% false positive rate accounts for these. + +## 3. Script: `radarr/auto_quality_switch.sh` + +### 3.1 Purpose + +Daily cron job that: +1. Fetches all movies from Radarr API +2. Computes the dynamic P95 threshold from movies with both web + physical dates +3. Identifies movies where: + - `digitalRelease` exists and `physicalRelease` is null + - Days since `digitalRelease` > computed threshold + - Current `qualityProfileId` matches the source profile +4. Optionally switches those movies to the target quality profile +5. Reports what happened + +### 3.2 Behavior modes + +| Mode | Trigger | Effect | +|------|---------|--------| +| Dry-run | Default or `-n` / `--dry-run` | Print candidates, no API mutations | +| Apply | `--apply` | Actually switch profiles | +| JSON | `-j` / `--json` | Output machine-readable JSON | +| Quiet | `-q` / `--quiet` | Only print errors and counts, no per-movie list | + +**Mode precedence:** + +1. `--apply` flag always overrides `DRY_RUN` config. This means: + - `DRY_RUN=true` in conf + `--apply` flag → **applies** (flag wins) + - `DRY_RUN=false` in conf + no flag → **applies** (conf says go) + - `DRY_RUN=false` in conf + `--dry-run` flag → **dry-run** (flag wins) +2. `--json` takes precedence over `--quiet` for output format. If both passed, JSON wins. +3. `--apply` + `--json` → switches profiles AND outputs JSON with results. + +### 3.3 Exit codes + +| Code | Meaning | +|------|---------| +| 0 | Success (no changes needed in dry-run) | +| 0 | Success (N movies switched in apply mode) | +| 1 | API error or configuration error | +| 127 | Missing executable dependency | + +## 4. Configuration + +### 4.1 From `scripts.conf` + +The script sources `scripts_common.sh` and reads these existing settings: + +```sh +RADARR_API_URL="http://ip:7878/api/v3" +RADARR_API_KEY="your_api_key" +``` + +### 4.2 Script-specific defaults (can be overridden in `scripts.conf`) + +| Variable | Default | Description | +|----------|---------|-------------| +| `SOURCE_PROFILE_NAME` | `Remux-2160p` | Profile name to match movies that need switching. Case-sensitive, must match Radarr exactly. | +| `TARGET_PROFILE_NAME` | `WebDL-2160p` | Profile name to switch matched movies to. Case-sensitive, must match Radarr exactly. | +| `P_VALUE` | `0.95` | Percentile used as threshold (0.0–1.0). P95 = 95% of physical releases happen within this window. Lower = more aggressive; higher = more conservative. | +| `MIN_SAMPLE` | `30` | Minimum movies with both web+physical dates needed to compute threshold. Below this, uses `FALLBACK_THRESHOLD`. | +| `FALLBACK_THRESHOLD` | `365` | Hardcoded threshold (days) when sample is too small for reliable percentiles. | +| `MIN_THRESHOLD` | `90` | Floor for computed threshold. Prevents nonsense when data is noisy. | +| `MAX_THRESHOLD` | `730` | Ceiling for computed threshold. Prevents excessively long waits. | +| `DRY_RUN` | `true` | Default preview mode. Set `false` in `scripts.conf` or pass `--apply` to execute switches. | +| `MAX_SWITCH_PER_RUN` | `0` | Max movies to switch per run. `0` = unlimited. Limits batch size to avoid hammering API. | +| `TRIGGER_SEARCH` | `true` | After switching, call `POST /api/v3/command` to queue `MoviesSearch` for switched movies. Set `false` to only switch profiles. | + +### 4.3 Example `scripts.conf` overrides + +```sh +# Auto quality switch settings +SOURCE_PROFILE_NAME="Remux-2160p" +TARGET_PROFILE_NAME="WebDL-2160p" +P_VALUE=0.95 +# Uncomment to allow switches without --apply flag: +# DRY_RUN=false +# Safety limit per run: +# MAX_SWITCH_PER_RUN=50 +# Uncomment to skip search after switch: +# TRIGGER_SEARCH=false +``` + +## 5. Algorithm + +### 5.1 Threshold computation + +``` +all_movies = GET /api/v3/movie +``` + +The IQR filter and percentile computation happen in a single jq pass. Pass `P_VALUE` and `MIN_SAMPLE` as jq args: + +```jq +# IQR filter function: removes outliers using 1.5*IQR rule +def iqr_filter: + if length == 0 then [] + else + sort as $sorted + | (($sorted | length) * 0.25 | floor) as $q1_idx + | (($sorted | length) * 0.75 | floor) as $q3_idx + | $sorted[$q1_idx] as $q1 + | $sorted[$q3_idx] as $q3 + | ($q3 - $q1) as $iqr + | ($q1 - 1.5 * $iqr) as $lower + | ($q3 + 1.5 * $iqr) as $upper + | [.[] | select(. >= $lower and . <= $upper)] + end; + +# Build gap array: movies with both digital + physical, gap in days +[.[] | select(.digitalRelease != null and .physicalRelease != null) + | ((.physicalRelease | fromdateiso8601) - (.digitalRelease | fromdateiso8601)) / 86400] as $gaps + +# If sample too small, return null (caller uses FALLBACK_THRESHOLD) +| if ($gaps | length) < ($min_sample | tonumber) then + {threshold: null, sample_size: ($gaps | length), used_fallback: true} + +# Otherwise: IQR filter, then percentile on filtered +else + ($gaps | iqr_filter | sort) as $filtered + | ($filtered[(($filtered | length) * ($p_value | tonumber)) | floor]) as $p_val + | {threshold: $p_val, sample_size: ($gaps | length), filtered_size: ($filtered | length), used_fallback: false} +end +``` + +**Percentile approximation:** Uses index-based percentile: `sorted[floor(length * p)]`. With 30 samples at P95, index 28 = ~96.7th percentile (slightly conservative). This is acceptable — the threshold is clamped by MIN/MAX anyway. + +**Clamping (in shell, not jq):** +```sh +_threshold=$(printf '%s' "${_threshold_json}" | jq -r '.threshold') +_used_fallback=$(printf '%s' "${_threshold_json}" | jq -r '.used_fallback') + +if [ "${_used_fallback}" = "true" ] || [ -z "${_threshold}" ] || [ "${_threshold}" = "null" ] +then + _threshold="${FALLBACK_THRESHOLD}" +fi + +# Clamp to [MIN_THRESHOLD, MAX_THRESHOLD] +[ "${_threshold}" -lt "${MIN_THRESHOLD}" ] && _threshold="${MIN_THRESHOLD}" +[ "${_threshold}" -gt "${MAX_THRESHOLD}" ] && _threshold="${MAX_THRESHOLD}" +``` + +### 5.2 Candidate matching + +``` +# Map profile names to IDs +profiles = GET /api/v3/qualityProfile +source_id = profiles[where name == SOURCE_PROFILE_NAME].id +target_id = profiles[where name == TARGET_PROFILE_NAME].id + +# Find candidates +candidates = [] +for movie in all_movies: + if movie.digitalRelease == null: + skip # No web date to measure from + if movie.physicalRelease != null: + skip # Already has physical + if movie.qualityProfileId != source_id: + skip # Already on a different profile + if movie.inCinemas == null: + skip # No cinema date (pre-release, not relevant) + if movie.hasFile == true: + skip # Already has a downloaded file + if movie.monitored == false: + skip # Unmonitored — search won't trigger anyway + + waiting_days = (now - movie.digitalRelease) in days + + if waiting_days >= threshold: + candidates.append(movie) +``` + +### 5.3 Profile switch + +``` +switched_ids = [] +for candidate in candidates: + PUT /api/v3/movie/editor + body: { + "movieIds": [candidate.id], + "qualityProfileId": target_id + } + switched_ids.append(candidate.id) +``` + +### 5.4 Search trigger + +After all profile switches, if `TRIGGER_SEARCH=true` and `switched_ids` is non-empty: + +``` +if TRIGGER_SEARCH and len(switched_ids) > 0: + POST /api/v3/command + body: { + "name": "MoviesSearch", + "movieIds": switched_ids + } +``` + +Radarr will queue search tasks for the switched movies. They get picked up by Radarr's built-in search queue and download matching releases — no external tool needed. + +**Success criteria:** HTTP 200 (Accepted) response with a JSON body containing a `jobId` field. Any other HTTP status or curl failure = error. Log the response body on failure for diagnosis. + +**Rate limiting:** The search command is a single API call regardless of how many movies were switched. Radarr handles the queue internally, no need to batch or delay. The initial profile switch calls are already rate-limited via `sleep 0.5` between them per `MAX_SWITCH_PER_RUN`. + +### 5.5 Safety guards + +1. **Always dry-run first** — default mode shows what would change +2. **Threshold clamping** — computed P95 clamped to `[MIN_THRESHOLD, MAX_THRESHOLD]` +3. **Minimum sample** — below `MIN_SAMPLE` movies with both dates, use `FALLBACK_THRESHOLD` +4. **Batch limiting** — `MAX_SWITCH_PER_RUN` caps API writes per invocation +5. **Profile existence check** — abort if source or target profile not found +6. **No-downgrade guarantee** — only switches from SOURCE → TARGET, never the reverse + +## 6. Output Format + +### 6.1 Pretty table (default) + +``` +Auto Quality Profile Switch +=========================== +Threshold: P95 = 265 days (based on 1921 movies with both dates) +Source profile: Remux-2160p (id: 5) +Target profile: WebDL-2160p (id: 7) + +Movie Waiting Current → Target +The Matrix Resurrections 487d Remux-2160p → WebDL-2160p +... + +DRY-RUN: 42 movies would switch. Run with --apply to execute. +``` + +Or in apply mode: + +``` +APPLY: Switched 42 movies to WebDL-2160p +QUEUED: 42 movies sent for search +Errors: 0 +``` + +### 6.2 JSON mode (`--json`) + +```json +{ + "threshold_days": 265, + "p_value": 0.95, + "sample_size": 1921, + "source_profile": {"id": 5, "name": "Remux-2160p"}, + "target_profile": {"id": 7, "name": "WebDL-2160p"}, + "candidates": [ + {"id": 123, "title": "The Matrix Resurrections", "waiting_days": 487, "year": 2021} + ], + "candidate_count": 42, + "switched_count": 42, + "searched_count": 42, + "search_triggered": true, + "dry_run": false +} +``` + +## 7. Implementation Plan + +### Phase 1: Scaffold (1 step) + +- Create `radarr/auto_quality_switch.sh` +- Changelog header following project convention +- Source `scripts_common.sh`, call `load_config` +- Define default variables (section 4.2) +- **Update `radarr/connect/scripts.conf.sample`** with all new config variables from section 4.2, commented out with defaults. Project convention requires sample files to document all config variables. + +### Phase 2: Profile resolution (1 step) + +- Implement `_resolve_profile_id(name)`: + - `GET /api/v3/qualityProfile` + - `jq` to find profile by name + - Exit with error if not found +- Validate both source and target profiles exist before proceeding + +### Phase 3: Threshold computation (1 step) + +- Implement the jq query from section 5.1 (inline above — do not reference external files) +- Pass `P_VALUE` and `MIN_SAMPLE` as `--arg` to jq +- Parse result: extract `threshold`, `sample_size`, `used_fallback` +- Apply fallback logic if `used_fallback == true` +- Clamp to `[MIN_THRESHOLD, MAX_THRESHOLD]` in shell +- Log: computed P-value, sample size, outliers removed, final threshold + +### Phase 4: Candidate matching (1 step) + +- Implement in a single jq pass: + - Filter to movies meeting all criteria + - Compute `waiting_days` using `now` + - Compare against threshold + - Output `id`, `title`, `year`, `waiting_days`, `qualityProfileId` + +### Phase 5: Switch execution (1 step) + +- Implement `switch_movie_profile(movie_id, target_id)`: + - Dry-run mode: log intent + - Apply mode: `PUT /api/v3/movie/editor` + - Rate-limit with `sleep 0.5` between calls + - Respect `MAX_SWITCH_PER_RUN` + - Collect switched movie IDs into a list + +### Phase 6: Search trigger (1 step) + +- After all switches, if `TRIGGER_SEARCH=true` and switched list non-empty: + - `POST /api/v3/command` with `{"name": "MoviesSearch", "movieIds": [...]}` + - Validate response for success/error + - Log number of movies queued for search + +### Phase 7: Output (1 step) + +- Pretty table printing: + - Summary header + - Per-movie candidate list using `printf` with fixed-width format specifiers (e.g., `%-40s %8s %-20s %s`). Do NOT use `column -t` — it is not available on all systems (notably some FreeBSD/minimal Linux images). + - Final count and mode notice +- JSON output via `--json` flag + +### Phase 8: Verification (1 step) + +- Run `shellcheck -e SC3043 -s sh` +- Test with `DRY_RUN=true` on a real Radarr instance +- Verify profile resolution with both existing and non-existing profile names +- Verify threshold output matches `release_date_stats.sh` P95 value + +## 8. Edge Cases + +| Edge case | Handling | +|-----------|----------| +| No movies with both dates | Use `FALLBACK_THRESHOLD`, warn | +| Source profile not found | Print configured name + available profiles, exit 1 | +| Target profile not found | Print configured name + available profiles, exit 1 | +| Source == Target | Print warning, skip all | +| Movie with null digitalRelease or missing dates | Skip (no web date to measure) | +| Movie already on target profile | Skip (already switched) | +| Movie already has a downloaded file (`hasFile == true`) | Skip (already has a release) | +| Movie is unmonitored (`monitored == false`) | Skip (search won't trigger) | +| API unreachable | Print error, exit 1 | +| All candidates already switched | "0 candidates" — success | +| P95 computed below `MIN_THRESHOLD` | Clamp to `MIN_THRESHOLD`, log adjustment | +| P95 computed above `MAX_THRESHOLD` | Clamp to `MAX_THRESHOLD`, log adjustment | +| `MAX_SWITCH_PER_RUN` reached mid-batch | Stop, report partial switch count. Still trigger search for movies switched so far. | +| Search command fails (API error) | Log warning, continue. Movies are already switched — they'll be picked up on next Radarr search pass anyway. | +| `TRIGGER_SEARCH=true` but no movies switched | Skip search call entirely. No-op. | + +## 9. Scheduling (user responsibility) + +Recommended crontab (daily at 6am): + +```cron +0 6 * * * /path/to/radarr/auto_quality_switch.sh --apply >> /var/log/quality-switch.log 2>&1 +``` + +Or run in dry-run mode for a week first to observe: + +```cron +0 6 * * * /path/to/radarr/auto_quality_switch.sh >> /var/log/quality-switch-preview.log 2>&1 +``` + +## 10. Dependencies + +- `curl` — API calls +- `jq` 1.6+ — JSON processing (`fromdateiso8601`, `now`, sort) +- Standard POSIX `sh` — runtime +- Radarr API access with key configured + +## 11. Project conventions (must follow) + +- Shebang: `#!/usr/bin/env sh` +- `# shellcheck disable=SC3043` for `local` +- Changelog version blocks (newest first) +- `scripts_common.sh` for API helpers +- `load_config "$(dirname "$0")/connect"` for config — note that `load_config` accepts an optional config directory as `$1`, defaulting to `$(dirname "$0")`. This script lives in `radarr/` (not `radarr/connect/`), so it MUST pass the path: `load_config "$(dirname "$0")/connect"`. See `radarr/research/release_date_stats.sh` for a working example of this pattern. +- Local vars prefixed with underscore +- Errors to stderr +- Quote all variable expansions +- `printf` for formatted output, `echo` for simple strings +- Indent 4 spaces, 100-char soft line limit diff --git a/radarr/auto_quality_switch.sh b/radarr/auto_quality_switch.sh new file mode 100644 index 0000000..6215b6b --- /dev/null +++ b/radarr/auto_quality_switch.sh @@ -0,0 +1,472 @@ +#!/usr/bin/env sh +# Dont warn on the word `local` +# shellcheck disable=SC3043 + +# Script to automatically switch movies from a Remux-only quality profile +# to a WebDL-enabled profile when no physical release appears within a +# statistically determined threshold (P95 of the web->physical gap). +# +# Requirements: +# * sh (tested with sh from FreeBSD base FreeBSD 14.1) +# * curl (tested with 8.10.1) +# * jq (tested with 1.7.1) +# +# Version 0.1.0 (Released 2026-07-02) +# * Initial implementation + +# Load shared library and configuration +. "$(dirname "$0")/connect/scripts_common.sh" +load_config "$(dirname "$0")/connect" + +# Script-specific defaults +: "${SOURCE_PROFILE_NAME:=Remux-2160p}" +: "${TARGET_PROFILE_NAME:=WebDL-2160p}" +: "${P_VALUE:=0.95}" +: "${MIN_SAMPLE:=30}" +: "${FALLBACK_THRESHOLD:=365}" +: "${MIN_THRESHOLD:=90}" +: "${MAX_THRESHOLD:=730}" +: "${DRY_RUN:=true}" +: "${MAX_SWITCH_PER_RUN:=0}" +: "${TRIGGER_SEARCH:=true}" +: "${DEBUG:=false}" + +# CLI flags +_FLAG_APPLY=false +_FLAG_JSON=false +_FLAG_QUIET=false +_DEFER_JSON=false + +while [ $# -gt 0 ]; do + case "$1" in + --apply) _FLAG_APPLY=true; shift ;; + -n|--dry-run) DRY_RUN=true; shift ;; + -j|--json) _FLAG_JSON=true; shift ;; + -q|--quiet) _FLAG_QUIET=true; shift ;; + -d|--debug) DEBUG=true; shift ;; + *) break ;; + esac +done + +check_needed_executables "curl jq" + +debug_log "=== Auto Quality Profile Switch ===" +debug_log "Source profile: ${SOURCE_PROFILE_NAME}" +debug_log "Target profile: ${TARGET_PROFILE_NAME}" +debug_log "P_VALUE: ${P_VALUE}" + +# --apply flag overrides DRY_RUN config +if [ "${_FLAG_APPLY}" = "true" ] +then + DRY_RUN=false +fi + +# --json takes precedence over --quiet +if [ "${_FLAG_JSON}" = "true" ] +then + _FLAG_QUIET=false +fi + +############################################################################## +# Phase 2: Profile resolution +############################################################################## + +_resolve_profile_id() { + local _profile_name _profiles _id + + _profile_name="$1" + + if [ -z "${_profile_name}" ] + then + echo "ERROR: resolve_profile_id called with empty name" >&2 + return 1 + fi + + _profiles=$(radarr_api_get "qualityProfile") + + if [ -z "${_profiles}" ] + then + echo "ERROR: No response from qualityProfile API" >&2 + return 1 + fi + + _id=$(printf '%s' "${_profiles}" | jq -r --arg name "${_profile_name}" \ + '[.[] | select(.name == $name)] | .[0].id // empty') + + if [ -z "${_id}" ] + then + echo "ERROR: Quality profile '${_profile_name}' not found" >&2 + echo "ERROR: Available profiles:" >&2 + printf '%s' "${_profiles}" | jq -r '.[].name' | while read -r _line + do + echo "ERROR: ${_line}" >&2 + done + return 1 + fi + + printf '%s' "${_id}" +} + +debug_log "Resolving source profile ID" +SOURCE_PROFILE_ID=$(_resolve_profile_id "${SOURCE_PROFILE_NAME}") +_resolve_rc=$? +if [ "${_resolve_rc}" -ne 0 ] +then + exit 1 +fi +debug_log "Source profile: ${SOURCE_PROFILE_NAME} (id: ${SOURCE_PROFILE_ID})" + +debug_log "Resolving target profile ID" +TARGET_PROFILE_ID=$(_resolve_profile_id "${TARGET_PROFILE_NAME}") +_resolve_rc=$? +if [ "${_resolve_rc}" -ne 0 ] +then + exit 1 +fi +debug_log "Target profile: ${TARGET_PROFILE_NAME} (id: ${TARGET_PROFILE_ID})" + +if [ "${SOURCE_PROFILE_ID}" = "${TARGET_PROFILE_ID}" ] +then + echo "ERROR: Source and target profiles are the same (id: ${SOURCE_PROFILE_ID})" >&2 + echo "ERROR: Nothing to switch. Exiting." >&2 + exit 1 +fi + +############################################################################## +# Phase 3: Threshold computation +############################################################################## + +debug_log "Fetching all movies from Radarr API" +_ALL_MOVIES=$(radarr_api_get "movie") + +if [ -z "${_ALL_MOVIES}" ] +then + echo "ERROR: No response from movie API" >&2 + exit 1 +fi + +_THRESHOLD_JSON=$(printf '%s' "${_ALL_MOVIES}" | jq \ + --arg p_value "${P_VALUE}" \ + --arg min_sample "${MIN_SAMPLE}" \ + ' +def iqr_filter: + if length == 0 then [] + else + sort as $sorted + | (($sorted | length) * 0.25 | floor) as $q1_idx + | (($sorted | length) * 0.75 | floor) as $q3_idx + | $sorted[$q1_idx] as $q1 + | $sorted[$q3_idx] as $q3 + | ($q3 - $q1) as $iqr + | ($q1 - 1.5 * $iqr) as $lower + | ($q3 + 1.5 * $iqr) as $upper + | [.[] | select(. >= $lower and . <= $upper)] + end; + +[.[] | select(.digitalRelease != null and .physicalRelease != null) + | ((.physicalRelease | fromdateiso8601) - (.digitalRelease | fromdateiso8601)) / 86400] as $gaps + +| if ($gaps | length) < ($min_sample | tonumber) then + {threshold: null, sample_size: ($gaps | length), used_fallback: true} +else + ($gaps | iqr_filter | sort) as $filtered + | ($filtered[(($filtered | length) * ($p_value | tonumber)) | floor]) as $p_val + | {threshold: $p_val, sample_size: ($gaps | length), filtered_size: ($filtered | length), used_fallback: false} +end +') + +_THRESHOLD=$(printf '%s' "${_THRESHOLD_JSON}" | jq -r '.threshold') +_SAMPLE_SIZE=$(printf '%s' "${_THRESHOLD_JSON}" | jq -r '.sample_size') +_USED_FALLBACK=$(printf '%s' "${_THRESHOLD_JSON}" | jq -r '.used_fallback') + +if [ "${_USED_FALLBACK}" = "true" ] || [ -z "${_THRESHOLD}" ] || [ "${_THRESHOLD}" = "null" ] +then + _THRESHOLD="${FALLBACK_THRESHOLD}" + debug_log "Using fallback threshold: ${_THRESHOLD}d (sample size: ${_SAMPLE_SIZE})" +fi + +# Clamp to [MIN_THRESHOLD, MAX_THRESHOLD] +if [ "${_THRESHOLD}" -lt "${MIN_THRESHOLD}" ] 2>/dev/null +then + _THRESHOLD="${MIN_THRESHOLD}" + debug_log "Threshold clamped to MIN_THRESHOLD: ${_THRESHOLD}d" +fi + +if [ "${_THRESHOLD}" -gt "${MAX_THRESHOLD}" ] 2>/dev/null +then + _THRESHOLD="${MAX_THRESHOLD}" + debug_log "Threshold clamped to MAX_THRESHOLD: ${_THRESHOLD}d" +fi + +############################################################################## +# Phase 4: Candidate matching +############################################################################## + +_CANDIDATES=$(printf '%s' "${_ALL_MOVIES}" | jq \ + --arg threshold "${_THRESHOLD}" \ + --arg source_id "${SOURCE_PROFILE_ID}" \ + ' +[.[] | select( + .digitalRelease != null + and .physicalRelease == null + and .qualityProfileId == ($source_id | tonumber) + and .inCinemas != null + and .hasFile == false + and .monitored == true +) | ((.digitalRelease | fromdateiso8601)) as $web_epoch + | ((now - $web_epoch) / 86400) as $waiting_days + | select($waiting_days >= ($threshold | tonumber)) + | {id: .id, title: .title, year: .year, waiting_days: $waiting_days, qualityProfileId: .qualityProfileId}] +| sort_by(.waiting_days) | reverse +') + +_CANDIDATE_COUNT=$(printf '%s' "${_CANDIDATES}" | jq 'length') + +# Unset to free memory +unset _ALL_MOVIES + +debug_log "Threshold: P${P_VALUE} = ${_THRESHOLD}d (based on ${_SAMPLE_SIZE} movies with both dates)" +debug_log "Candidates: ${_CANDIDATE_COUNT} movies" + +############################################################################## +# Phase 7: Output (table or JSON) +############################################################################## + +if [ "${_FLAG_JSON}" = "true" ] +then + _DEFER_JSON=false + + if [ "${DRY_RUN}" = "true" ] || [ "${_CANDIDATE_COUNT}" -eq 0 ] + then + # Dry-run or no candidates: output JSON immediately and exit + _SWITCHED_COUNT=0 + + printf '%s' "${_CANDIDATES}" | jq \ + --arg threshold "${_THRESHOLD}" \ + --arg p_value "${P_VALUE}" \ + --arg sample_size "${_SAMPLE_SIZE}" \ + --arg src_name "${SOURCE_PROFILE_NAME}" \ + --arg src_id "${SOURCE_PROFILE_ID}" \ + --arg tgt_name "${TARGET_PROFILE_NAME}" \ + --arg tgt_id "${TARGET_PROFILE_ID}" \ + --arg switched_count "${_SWITCHED_COUNT}" \ + --arg searched_count "0" \ + --arg search_triggered "false" \ + --argjson dry_run "${DRY_RUN}" \ + ' +{ + threshold_days: ($threshold | tonumber), + p_value: ($p_value | tonumber), + sample_size: ($sample_size | tonumber), + source_profile: {id: ($src_id | tonumber), name: $src_name}, + target_profile: {id: ($tgt_id | tonumber), name: $tgt_name}, + candidates: ., + candidate_count: length, + switched_count: ($switched_count | tonumber), + searched_count: ($searched_count | tonumber), + search_triggered: ($search_triggered == "true"), + dry_run: $dry_run +} +' + echo + exit 0 + fi + + # Apply + JSON mode: defer JSON output until after switch + _DEFER_JSON=true +fi + +# Print pretty table (only in non-JSON mode) +if [ "${_FLAG_JSON}" = "false" ] +then + if [ "${_FLAG_QUIET}" = "false" ] + then + echo + echo "Auto Quality Profile Switch" + echo "===========================" + echo + echo "Threshold: P${P_VALUE} = ${_THRESHOLD}d (based on ${_SAMPLE_SIZE} movies with both dates)" + echo "Source profile: ${SOURCE_PROFILE_NAME} (id: ${SOURCE_PROFILE_ID})" + echo "Target profile: ${TARGET_PROFILE_NAME} (id: ${TARGET_PROFILE_ID})" + echo + fi + + if [ "${_CANDIDATE_COUNT}" -gt 0 ] && [ "${_FLAG_QUIET}" = "false" ] + then + printf '%-50s %8s %-20s %s\n' "Movie" "Waiting" "Current Profile" "-> Target" + printf '%-50s %8s %-20s %s\n' "-----" "-------" "---------------" "--------" + + printf '%s' "${_CANDIDATES}" | jq -r '.[] | "\(.title) (\(.year))|\(.waiting_days)|\(.qualityProfileId)"' | \ + while IFS='|' read -r _title _waiting _profile_id; do + printf '%-50s %6dd %-20s -> %s\n' "${_title}" "${_waiting%.*}" "${SOURCE_PROFILE_NAME}" "${TARGET_PROFILE_NAME}" + done + echo + fi + + if [ "${_FLAG_QUIET}" = "false" ] + then + if [ "${_CANDIDATE_COUNT}" -eq 0 ] + then + echo "No candidates to switch." + echo + exit 0 + fi + + if [ "${DRY_RUN}" = "true" ] + then + echo "DRY-RUN: ${_CANDIDATE_COUNT} movies would switch. Run with --apply to execute." + echo + exit 0 + fi + fi +fi + +############################################################################## +# Phase 5: Switch execution +############################################################################## + +_SWITCHED_COUNT=0 +_SWITCHED_IDS="" + +_SWITCH_TEMP=$(mktemp) +# shellcheck disable=SC2064 +trap 'rm -f "${_SWITCH_TEMP}"; exit 130' INT TERM +trap 'rm -f "${_SWITCH_TEMP}"' EXIT + +printf '%s' "${_CANDIDATES}" | jq -r '.[].id' > "${_SWITCH_TEMP}" + +while read -r _movie_id +do + if [ "${MAX_SWITCH_PER_RUN}" -gt 0 ] && [ "${_SWITCHED_COUNT}" -ge "${MAX_SWITCH_PER_RUN}" ] + then + debug_log "MAX_SWITCH_PER_RUN (${MAX_SWITCH_PER_RUN}) reached, stopping" + break + fi + + _payload=$(printf '{"movieIds": [%s], "qualityProfileId": %s}' "${_movie_id}" "${TARGET_PROFILE_ID}") + + debug_log "Switching movie ${_movie_id} to profile ${TARGET_PROFILE_ID}" + + _response=$(curl \ + -s \ + -X PUT \ + -H "Accept-Encoding: application/json" \ + -H "X-Api-Key: ${RADARR_API_KEY}" \ + -H "Content-Type: application/json" \ + -d "${_payload}" \ + "${RADARR_API_URL}/movie/editor" \ + -w "\n%{http_code}") + + _http_code=$(printf '%s' "${_response}" | tail -1) + _body=$(printf '%s' "${_response}" | sed '$d') + + case "${_http_code}" in + 2*) + _SWITCHED_COUNT=$((_SWITCHED_COUNT + 1)) + if [ -z "${_SWITCHED_IDS}" ] + then + _SWITCHED_IDS="${_movie_id}" + else + _SWITCHED_IDS="${_SWITCHED_IDS},${_movie_id}" + fi + debug_log " OK (HTTP ${_http_code})" + ;; + *) + echo "WARN: Failed to switch movie ${_movie_id} (HTTP ${_http_code}): ${_body}" >&2 + ;; + esac + + sleep 0.5 +done < "${_SWITCH_TEMP}" + +rm -f "${_SWITCH_TEMP}" +trap - INT TERM EXIT + +############################################################################## +# Phase 6: Search trigger +############################################################################## + +_SEARCH_QUEUED=0 + +if [ "${TRIGGER_SEARCH}" = "true" ] && [ "${_SWITCHED_COUNT}" -gt 0 ] +then + debug_log "Triggering search for ${_SWITCHED_COUNT} switched movies" + + _ids_json=$(printf '%s' "${_SWITCHED_IDS}" | jq -R 'split(",") | map(tonumber)') + + _search_response=$(curl \ + -s \ + -X POST \ + -H "Accept-Encoding: application/json" \ + -H "X-Api-Key: ${RADARR_API_KEY}" \ + -H "Content-Type: application/json" \ + -d "{\"name\": \"MoviesSearch\", \"movieIds\": ${_ids_json}}" \ + "${RADARR_API_URL}/command" \ + -w "\n%{http_code}") + + _search_http=$(printf '%s' "${_search_response}" | tail -1) + _search_body=$(printf '%s' "${_search_response}" | sed '$d') + + case "${_search_http}" in + 2*) + _SEARCH_QUEUED="${_SWITCHED_COUNT}" + debug_log "Search command queued (HTTP ${_search_http}, jobId: $(printf '%s' "${_search_body}" | jq -r '.jobId // "unknown"'))" + ;; + *) + echo "WARN: Search command failed (HTTP ${_search_http}): ${_search_body}" >&2 + echo "WARN: Movies are switched but not searched. Next Radarr search pass will pick them up." >&2 + ;; + esac +fi + +############################################################################## +# Final summary +############################################################################## + +if [ "${_DEFER_JSON}" = "true" ] +then + # JSON output with real switched_count and searched_count + printf '%s' "${_CANDIDATES}" | jq \ + --arg threshold "${_THRESHOLD}" \ + --arg p_value "${P_VALUE}" \ + --arg sample_size "${_SAMPLE_SIZE}" \ + --arg src_name "${SOURCE_PROFILE_NAME}" \ + --arg src_id "${SOURCE_PROFILE_ID}" \ + --arg tgt_name "${TARGET_PROFILE_NAME}" \ + --arg tgt_id "${TARGET_PROFILE_ID}" \ + --arg switched_count "${_SWITCHED_COUNT}" \ + --arg searched_count "${_SEARCH_QUEUED}" \ + --argjson dry_run false \ + ' +{ + threshold_days: ($threshold | tonumber), + p_value: ($p_value | tonumber), + sample_size: ($sample_size | tonumber), + source_profile: {id: ($src_id | tonumber), name: $src_name}, + target_profile: {id: ($tgt_id | tonumber), name: $tgt_name}, + candidates: ., + candidate_count: length, + switched_count: ($switched_count | tonumber), + searched_count: ($searched_count | tonumber), + search_triggered: (($searched_count | tonumber) > 0), + dry_run: false +} +' + echo +elif [ "${_FLAG_QUIET}" = "false" ] +then + echo "APPLY: Switched ${_SWITCHED_COUNT} movies to ${TARGET_PROFILE_NAME}" + if [ "${_SEARCH_QUEUED}" -gt 0 ] + then + echo "QUEUED: ${_SEARCH_QUEUED} movies sent for search" + elif [ "${TRIGGER_SEARCH}" = "true" ] && [ "${_SWITCHED_COUNT}" -gt 0 ] + then + echo "WARN: Search was not queued (see warnings above)" + elif [ "${TRIGGER_SEARCH}" = "false" ] + then + echo "Search skipped (TRIGGER_SEARCH=false)" + fi + echo +fi + +exit 0 diff --git a/radarr/connect/scripts.conf.sample b/radarr/connect/scripts.conf.sample index 84444fc..3825ed6 100644 --- a/radarr/connect/scripts.conf.sample +++ b/radarr/connect/scripts.conf.sample @@ -40,3 +40,29 @@ DRY_RUN="false" # Debug mode. When true, prints detailed diagnostic messages. DEBUG="false" + +# --- Auto Quality Switch settings (optional) --- +# Quality profile names for the auto quality switch script. +# SOURCE_PROFILE_NAME="Remux-2160p" +# TARGET_PROFILE_NAME="WebDL-2160p" + +# Percentile threshold for web->physical gap (0.0-1.0). +# P95 = wait until 95% of physical releases have appeared. +# P_VALUE="0.95" + +# Minimum sample of movies with both dates needed to compute threshold. +# Below this, FALLBACK_THRESHOLD is used. +# MIN_SAMPLE="30" + +# Hardcoded fallback threshold (days) when sample is too small. +# FALLBACK_THRESHOLD="365" + +# Clamping bounds for computed threshold. +# MIN_THRESHOLD="90" +# MAX_THRESHOLD="730" + +# Max movies to switch per run. 0 = unlimited. +# MAX_SWITCH_PER_RUN="0" + +# Trigger Radarr search after profile switch. +# TRIGGER_SEARCH="true" diff --git a/radarr/research/release_date_stats.sh b/radarr/research/release_date_stats.sh new file mode 100644 index 0000000..b02278d --- /dev/null +++ b/radarr/research/release_date_stats.sh @@ -0,0 +1,235 @@ +#!/usr/bin/env sh +# Dont warn on the word `local` +# shellcheck disable=SC3043 + +# Script to compute release date statistics from Radarr. +# Analyzes cinema, digital, and physical release dates across all movies. +# +# Requirements: +# * sh (tested with sh from FreeBSD base FreeBSD 14.1) +# * curl (tested with 8.10.1) +# * jq (tested with 1.7.1) +# +# Version 0.3.0 (Released 2026-07-02) +# * Add percentiles (P50/P90/P95/P99) to web->physical gap +# * Cinema+Web category shows waiting time distribution +# * Add threshold-based counts for profile switch planning +# * Decision-oriented output focused on profile policy +# +# Version 0.2.0 (Released 2026-07-02) +# * Add IQR-based outlier filtering for clean stats +# * Show raw + filtered side by side in output +# +# Version 0.1.0 (Released 2026-07-02) +# * Initial implementation +# * Fetch all movies from Radarr API +# * Compute web-to-physical gap stats (min/max/avg) +# * Count movies in 3 date-availability categories +# * Compute cinema-to-web and cinema-to-physical gaps per category +# * Pretty table + JSON output + +# Load shared library and configuration +. "$(dirname "$0")/../connect/scripts_common.sh" +load_config "$(dirname "$0")/../connect" + +: "${OUTPUT_JSON:=false}" +: "${DEBUG:=false}" +: "${PRETTY_JSON:=false}" + +# Parse optional flags +while [ $# -gt 0 ]; do + case "$1" in + -j|--json) OUTPUT_JSON=true; shift ;; + -p|--pretty) PRETTY_JSON=true; shift ;; + -d|--debug) DEBUG=true; shift ;; + *) break ;; + esac +done + +check_needed_executables "curl jq" + +debug_log "Fetching all movies from Radarr API" +_all_movies=$(radarr_api_get "movie") + +if [ -z "${_all_movies}" ] +then + echo "ERROR: No response from Radarr API" >&2 + exit 1 +fi + +_computed_stats=$(printf '%s' "${_all_movies}" | jq ' +def percentiles: + if length == 0 then null + else + sort as $sorted + | {p50: $sorted[(length * 0.5) | floor], + p90: $sorted[(length * 0.9) | floor], + p95: $sorted[(length * 0.95) | floor], + p99: $sorted[(length * 0.99) | floor]} + end; + +def iqr_filtered_stats: + if length == 0 then + {count: 0, raw: null, filtered: null} + else + sort as $sorted + | (($sorted | length) * 0.25 | floor) as $q1_idx + | (($sorted | length) * 0.75 | floor) as $q3_idx + | $sorted[$q1_idx] as $q1 + | $sorted[$q3_idx] as $q3 + | ($q3 - $q1) as $iqr + | ($q1 - 1.5 * $iqr) as $lower + | ($q3 + 1.5 * $iqr) as $upper + | [.[] | select(. >= $lower and . <= $upper)] as $filtered + | {count: length, + raw: {min_days: min, max_days: max, avg_days: (add / length)}, + filtered: (if ($filtered | length) > 0 then + {min_days: ($filtered | min), max_days: ($filtered | max), avg_days: (($filtered | add) / ($filtered | length)), outliers_removed: (length - ($filtered | length)), + percentiles: ($filtered | percentiles)} + else + {min_days: null, max_days: null, avg_days: null, outliers_removed: length, + percentiles: null} + end)} + end; + +map(select(.inCinemas != null)) +| { + "total_movies_with_cinema_date": length, + "web_to_physical": ( + [.[] | select(.digitalRelease != null and .physicalRelease != null) + | ((.physicalRelease | fromdateiso8601) - (.digitalRelease | fromdateiso8601)) / 86400] + | iqr_filtered_stats + ), + "cinema_only": ( + [.[] | select(.digitalRelease == null and .physicalRelease == null)] + | {count: length} + ), + "cinema_web": ( + [.[] | select(.digitalRelease != null and .physicalRelease == null) + | ((.digitalRelease | fromdateiso8601)) as $web_epoch + | ((now - $web_epoch) / 86400)] as $waiting_times + | ($waiting_times | sort) as $sorted + | if ($waiting_times | length) > 0 then + {count: ($waiting_times | length), + waiting_days_p50: $sorted[($sorted | length * 0.5) | floor], + waiting_days_p90: $sorted[($sorted | length * 0.9) | floor], + waiting_days_p95: $sorted[($sorted | length * 0.95) | floor], + waiting_days_max: $sorted[-1], + thresholds: { + gt_90: ([$waiting_times[] | select(. > 90)] | length), + gt_180: ([$waiting_times[] | select(. > 180)] | length), + gt_365: ([$waiting_times[] | select(. > 365)] | length), + gt_730: ([$waiting_times[] | select(. > 730)] | length) + }} + else + {count: 0} + end + ), + "cinema_physical": ( + [.[] | select(.digitalRelease == null and .physicalRelease != null) + | ((.physicalRelease | fromdateiso8601) - (.inCinemas | fromdateiso8601)) / 86400] + | iqr_filtered_stats + ) +} +') + +if [ -z "${_computed_stats}" ] +then + echo "ERROR: Failed to compute statistics" >&2 + exit 1 +fi + +if [ "${OUTPUT_JSON}" = "true" ] +then + if [ "${PRETTY_JSON}" = "true" ] + then + printf '%s' "${_computed_stats}" | jq '.' + else + printf '%s' "${_computed_stats}" + fi + echo + exit 0 +fi + +# Extract stats +_web_count=$(printf '%s' "${_computed_stats}" | jq '.web_to_physical.count') +_web_outliers=$(printf '%s' "${_computed_stats}" | jq -r '.web_to_physical.filtered.outliers_removed // 0') +_web_p50=$(printf '%s' "${_computed_stats}" | jq -r '.web_to_physical.filtered.percentiles.p50 // "N/A"') +_web_p90=$(printf '%s' "${_computed_stats}" | jq -r '.web_to_physical.filtered.percentiles.p90 // "N/A"') +_web_p95=$(printf '%s' "${_computed_stats}" | jq -r '.web_to_physical.filtered.percentiles.p95 // "N/A"') +_web_p99=$(printf '%s' "${_computed_stats}" | jq -r '.web_to_physical.filtered.percentiles.p99 // "N/A"') + +_cat_a=$(printf '%s' "${_computed_stats}" | jq '.cinema_only.count') + +_cat_b_count=$(printf '%s' "${_computed_stats}" | jq '.cinema_web.count') +_cat_b_wait_p50=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_web.waiting_days_p50 // "N/A"') +_cat_b_wait_p90=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_web.waiting_days_p90 // "N/A"') +_cat_b_wait_p95=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_web.waiting_days_p95 // "N/A"') +_cat_b_wait_max=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_web.waiting_days_max // "N/A"') +_cat_b_gt_90=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_web.thresholds.gt_90 // 0') +_cat_b_gt_180=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_web.thresholds.gt_180 // 0') +_cat_b_gt_365=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_web.thresholds.gt_365 // 0') +_cat_b_gt_730=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_web.thresholds.gt_730 // 0') + +_cat_c_count=$(printf '%s' "${_computed_stats}" | jq '.cinema_physical.count') +_cat_c_raw_min=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_physical.raw.min_days // "N/A"') +_cat_c_raw_max=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_physical.raw.max_days // "N/A"') +_cat_c_raw_avg=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_physical.raw.avg_days // "N/A"') +_cat_c_filt_min=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_physical.filtered.min_days // "N/A"') +_cat_c_filt_max=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_physical.filtered.max_days // "N/A"') +_cat_c_filt_avg=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_physical.filtered.avg_days // "N/A"') +_cat_c_filt_removed=$(printf '%s' "${_computed_stats}" | jq -r '.cinema_physical.filtered.outliers_removed // 0') + +_total=$(printf '%s' "${_computed_stats}" | jq '.total_movies_with_cinema_date') + +echo +echo "Release Date Analysis for Profile Decision" +echo "===========================================" +echo +echo "Movies with cinema date: ${_total}" + +echo +echo "=== Web -> Physical Gap ===" +echo "Movies with both dates: ${_web_count}" +echo "(IQR filtering removed ${_web_outliers} outliers)" +echo +echo "Days from web release to physical release:" +echo " P50 (median): ${_web_p50}d - half of physical releases within this" +echo " P90: ${_web_p90}d - 90% within this" +echo " P95: ${_web_p95}d - 95% within this" +echo " P99: ${_web_p99}d - 99% within this" + +echo +echo "=== Cinema + Web, No Physical ===" +echo "Movies: ${_cat_b_count}" +echo +echo "How long they have been waiting for a physical:" +if [ "${_cat_b_count}" -gt 0 ] 2>/dev/null +then + echo " P50 waiting: ${_cat_b_wait_p50}d" + echo " P90 waiting: ${_cat_b_wait_p90}d" + echo " P95 waiting: ${_cat_b_wait_p95}d" + echo + echo "Would switch profile at threshold:" + echo " > 90d: ${_cat_b_gt_90} movies" + echo " > 180d: ${_cat_b_gt_180} movies" + echo " > 365d: ${_cat_b_gt_365} movies" + echo " > 730d: ${_cat_b_gt_730} movies" +fi + +echo +echo "=== Cinema only (no web, no physical) ===" +echo "Movies: ${_cat_a}" + +echo +echo "=== Cinema + Physical, No Web ===" +echo "Movies: ${_cat_c_count}" +if [ "${_cat_c_count}" -gt 0 ] 2>/dev/null +then + echo " Raw: min ${_cat_c_raw_min}d max ${_cat_c_raw_max}d avg ${_cat_c_raw_avg}d" + echo " Filtered: min ${_cat_c_filt_min}d max ${_cat_c_filt_max}d avg ${_cat_c_filt_avg}d (${_cat_c_filt_removed} outliers removed)" +fi + +echo +echo "--- JSON ---" +printf '%s' "${_computed_stats}" | jq '.'