AI tutor, macaque templates, reliability calibration + free open-source tutor default

.env.template

@@ -7,3 +7,28 @@ OAUTHLIB_INSECURE_TRANSPORT=0 # Set this to 1 if in dev environment
 SECRET_KEY=this-really-needs-to-be-changed
 SQLALCHEMY_TRACK_MODIFICATIONS=False
 UPLOAD_DIR="uploads/"
+
+# ── Guided Learning tutor (LLM) ───────────────────────────────────────────────
+# Optional shared default that powers the /learn AI tutor when a visitor has not
+# entered their own key. Any OpenAI-compatible endpoint works. Visitors can
+# always override these from the in-page Settings panel (their key stays in
+# their browser). If none is set here and no visitor key is provided, the tutor
+# streams static reference material instead of failing.
+#
+# DEFAULT (free, open-source model, no paid key): Groq's Llama 3.3 70B. When a
+# key is present, LLM_BASE_URL and LLM_MODEL default to Groq automatically — so
+# create a free key at https://console.groq.com/keys and set ONLY the key:
+#   LLM_API_KEY=gsk_...
+# To use a different provider, also set LLM_BASE_URL / LLM_MODEL, e.g.:
+#   OpenAI   : LLM_BASE_URL=https://api.openai.com/v1      LLM_MODEL=gpt-4o-mini  (paid)
+#   Ollama   : LLM_BASE_URL=http://localhost:11434/v1      LLM_MODEL=llama3.2     (local, no key)
+LLM_API_KEY=
+LLM_BASE_URL=
+LLM_MODEL=
+# Embedding model for RAG knowledge retrieval (defaults follow LLM_API_KEY).
+# NOTE: Groq serves chat models only, not embeddings — do NOT run
+# `flask ingest-knowledge` against a Groq key. Retrieval degrades gracefully to
+# the built-in landmark reference when no embeddings store is present, so the
+# tutor works fine without ingestion. For real RAG embeddings use an
+# embeddings-capable provider (OpenAI, or local Ollama with nomic-embed-text).
+EMBED_MODEL=

.github/workflows/deploy.yml

@@ -20,6 +20,13 @@ jobs:
       - name: Checkout reference branch
         uses: actions/checkout@v4
 
+      - name: Setup Python
+        # Pin to a version inside pyproject's supported range (>=3.8,<3.12);
+        # ubuntu-latest now defaults to 3.12, which poetry would reject.
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
       - name: Install poetry
         uses: snok/install-poetry@v1
         with:
@@ -41,12 +48,25 @@ jobs:
           ORCID_OAUTH_CLIENT_ID: ${{ secrets.PRODUCTION_ORCID_OAUTH_CLIENT_ID }}
           ORCID_OAUTH_CLIENT_SECRET: ${{ secrets.PRODUCTION_ORCID_OAUTH_CLIENT_SECRET }}
           SECRET_KEY: ${{ secrets.PRODUCTION_SECRET_KEY }}
+          # Guided-learning tutor. Only the key is a secret; URL/model default to
+          # Groq's free Llama 3.3 70B endpoint but can be overridden by optional
+          # secrets to switch providers without editing this file.
+          LLM_API_KEY: ${{ secrets.PRODUCTION_LLM_API_KEY }}
+          LLM_BASE_URL: ${{ secrets.PRODUCTION_LLM_BASE_URL || 'https://api.groq.com/openai/v1' }}
+          LLM_MODEL: ${{ secrets.PRODUCTION_LLM_MODEL || 'llama-3.3-70b-versatile' }}
         run: |
           echo DATABASE_URL="$DATABASE_URL" >> .env
           echo FLASK_ENV="$FLASK_ENV" >> .env
           echo ORCID_OAUTH_CLIENT_ID="$ORCID_OAUTH_CLIENT_ID" >> .env
           echo ORCID_OAUTH_CLIENT_SECRET="$ORCID_OAUTH_CLIENT_SECRET" >> .env
           echo SECRET_KEY="$SECRET_KEY" >> .env
+          # Only write LLM vars when a key is configured, so a deploy without the
+          # secret leaves the tutor on its graceful static-reference fallback.
+          if [ -n "$LLM_API_KEY" ]; then
+            echo LLM_API_KEY="$LLM_API_KEY" >> .env
+            echo LLM_BASE_URL="$LLM_BASE_URL" >> .env
+            echo LLM_MODEL="$LLM_MODEL" >> .env
+          fi
 
       - name: Build release
         run: |

.gitignore

@@ -9,6 +9,14 @@ _site/
 *.pyc
 .env
 *.egg-info
+*.db
+~$*
+
+# Flask instance folder (local dev DB, cached templates)
+instance/
 
 # Workspaces
 afids.code-workspace
+
+# Generated Word manuscript (regenerate: python make_paper_doc.py)
+*.docx

DEPLOY_HANDOFF.md

@@ -0,0 +1,127 @@
+# AFIDs Validator — Deploy Handoff: AI Tutor (`/learn`) + Groq
+
+**Goal:** deploy the new **AI-tutor branch** to the live site **https://validator.afids.io**, with the guided-learning tutor (`/learn`) backed by a shared Groq API key so any visitor can use it without their own key.
+
+**Who this is for:** whoever currently has **SSH/admin access to the production server**. The code and the app-layer wiring are done and tested; the only remaining work is server-side deployment, which requires access this document's author does not have.
+
+---
+
+## 1. What's being deployed
+
+- **Repo:** `github.com/afids/afids-validator`
+- **Branch:** `enh/ai-tutor-learning-mode` (open PR **#253** → `master`, 17 commits ahead, mergeable)
+- **What it adds:**
+  - `/learn` guided-learning AI tutor (`afidsvalidator/learn.py`, `llm.py`, `templates/learn.html`)
+  - One **additive**, non-destructive DB migration: a `knowledge_chunks` table (`migrations/versions/c4e7f1a9b2d8_.py`)
+  - Macaque templates + reliability-calibration features
+- **Status:** code complete and pushed; tutor→Groq path tested locally and works (streams live Llama 3.3 70B). Nothing is deployed to production yet — `/learn` currently 404s on the live site.
+
+> **Note on CI:** PR #253's checks show red only because the repo uses `pull_request_target`, which runs the workflow definition from `master` (still has an old Python 3.8 matrix). This does **not** affect the deploy — the deploy workflow is manual (`workflow_dispatch`) and runs on whatever branch you point it at. You do **not** need CI green or a merge to deploy.
+
+---
+
+## 2. How production deploys work (important — not Docker)
+
+The live site is **not** run via `docker compose` (that's local-dev only). Production is a **wheel + venv + systemd** setup, deployed by a GitHub Actions workflow (`.github/workflows/deploy.yml`) that SSHes into the server. On trigger it:
+
+1. Builds a Python wheel from the chosen branch (`poetry build`).
+2. Writes a production `.env` from **GitHub Actions Secrets**.
+3. Runs `fabrictasks.py`, which SSHes into the server and:
+   - copies the wheel + `.env` into `/opt/afidsvalidator/releases/afidsvalidator-<timestamp>/`
+   - flips the `/opt/afidsvalidator/current` symlink to the new release
+   - `pip install <wheel>[deploy]` into `/opt/afidsvalidator/venv-afidsvalidator`
+   - runs `flask db upgrade` (applies the migration)
+   - `sudo systemctl restart afidsvalidator.service`
+
+So a deploy = **build wheel → ship over SSH → migrate DB → restart service.**
+
+---
+
+## 3. Prerequisites on the production server
+
+Confirm these are true on the live box before deploying:
+
+- [ ] SSH reachable; the deploy user's `~/.ssh/authorized_keys` contains the public key whose private half will be stored as the `PRIVATE_KEY` secret (see §4).
+- [ ] Directory layout exists: `/opt/afidsvalidator/releases/` and venv `/opt/afidsvalidator/venv-afidsvalidator/`.
+- [ ] **Passwordless sudo** for the deploy user (the workflow runs `sudo systemctl restart afidsvalidator.service`; it will hang on a password prompt).
+- [ ] `afidsvalidator.service` systemd unit exists and serves the app from the `current` symlink (already true — the site serves today).
+- [ ] Postgres running and reachable at the `DATABASE_URL` you'll put in the secrets.
+
+Quick verification commands (run on the box / from your machine):
+
+```bash
+ls -ld /opt/afidsvalidator/releases /opt/afidsvalidator/venv-afidsvalidator
+/opt/afidsvalidator/venv-afidsvalidator/bin/python --version      # note this version (see gotcha in §6)
+sudo -n systemctl status afidsvalidator.service >/dev/null && echo "passwordless sudo OK"
+ssh -i <your_private_key> <deploy_user>@<host> 'echo connected'
+```
+
+---
+
+## 4. GitHub Actions Secrets
+
+Set these on the repo (Settings → Secrets and variables → Actions, or the `gh secret set` commands below). **Secret values are entered at a hidden prompt or read from a file — never inline, never committed.**
+
+| Secret | What it is | Likely action |
+|---|---|---|
+| `PRODUCTION_URL` | SSH deploy target `user@host` (e.g. `ubuntu@<ip>`) | **Update** — current value is from 2023 and points at a decommissioned server. |
+| `PRIVATE_KEY` | SSH **private** key whose public half is in the box's `authorized_keys` | **Update** — must be a key the current box trusts. |
+| `PRODUCTION_DATABASE_URL` | Postgres connection string on the live box | **Verify** it matches the current box (see ⚠️ in §6). |
+| `PRODUCTION_SECRET_KEY` | Flask session secret | **Verify** it matches the current box. |
+| `PRODUCTION_FLASK_ENV` | e.g. `production` | Verify. |
+| `PRODUCTION_ORCID_OAUTH_CLIENT_ID` / `_SECRET` | ORCID login creds | Verify (only affects user login). |
+| `PRODUCTION_LLM_API_KEY` | **Groq API key** for the shared tutor | **Already set.** Optionally replace with a production-only key so it can be rotated independently of local dev. |
+
+The tutor's provider URL/model are **not** secrets — `deploy.yml` defaults them to Groq's Llama 3.3 70B (`https://api.groq.com/openai/v1`, `llama-3.3-70b-versatile`). The deploy is fail-safe: if `PRODUCTION_LLM_API_KEY` is empty, the LLM vars simply aren't written and the tutor falls back to static reference text instead of erroring.
+
+Commands (run from a clone of the repo):
+
+```bash
+gh secret set PRODUCTION_URL                       # paste user@host at prompt
+gh secret set PRIVATE_KEY < /path/to/deploy_private_key
+# only if the 2023 values no longer match the current box:
+gh secret set PRODUCTION_DATABASE_URL
+gh secret set PRODUCTION_SECRET_KEY
+# to use a prod-specific Groq key (optional):
+gh secret set PRODUCTION_LLM_API_KEY               # paste gsk_... at prompt
+
+gh secret list                                     # confirm names + timestamps
+```
+
+To get a free Groq key (if you'd rather use your own): https://console.groq.com/keys — free tier, no credit card. It only needs read access to chat models.
+
+---
+
+## 5. Deploy and verify
+
+```bash
+# Trigger against the tutor branch (or 'master' if you merge #253 first)
+gh workflow run "AFIDs Validator Deploy" --ref enh/ai-tutor-learning-mode
+
+# Watch — the make-or-break step is "Deploy release" (the SSH step)
+gh run watch $(gh run list --workflow=deploy.yml --limit 1 --json databaseId -q '.[0].databaseId')
+```
+
+Verify on the live site:
+
+- [ ] `https://validator.afids.io/learn` loads (no 404).
+- [ ] The tutor returns a **live streamed** response (confirms the Groq key landed) — not just static text.
+- [ ] A normal FCSV upload still validates (confirms the migration + restart were clean).
+
+---
+
+## 6. Gotchas (please read)
+
+1. **⚠️ Stale secrets can break a "successful" deploy.** The deploy **overwrites** the server `.env` from the GitHub Secrets. Most non-LLM secrets were set in **2023** (before the server was re-hosted). If `PRODUCTION_DATABASE_URL` / `PRODUCTION_SECRET_KEY` no longer match the current box, the deploy will run "green" but leave the site pointing at the wrong DB / invalidate sessions. **Before deploying, read the current `/opt/afidsvalidator/current/.env` on the box and make the GitHub Secrets match it** (then add the LLM key).
+
+2. **Python version hardcode.** `fabrictasks.py` line ~34 hardcodes the migrations path as `.../lib/python3.8/site-packages/migrations`. If the production venv is **not** Python 3.8, change `python3.8` in that line to the venv's actual version, or the `flask db upgrade` step fails. Check with `/opt/afidsvalidator/venv-afidsvalidator/bin/python --version`.
+
+3. **RAG ingestion is NOT required.** Do **not** run `flask ingest-knowledge` against the Groq key — Groq serves chat models only, no embeddings. The tutor degrades gracefully to built-in landmark reference material without an embeddings store, so it works fine with no ingestion.
+
+4. **The `Dockerfile` bit-rot is irrelevant to this deploy.** Production builds a wheel, not a Docker image, so the (currently broken) Dockerfile does not block deployment. It only affects local `docker compose`.
+
+---
+
+## 7. One-line summary
+
+Point `PRODUCTION_URL` + `PRIVATE_KEY` at the current server, make the 2023 DB/secret values match what's actually on the box, confirm the venv's Python version for the migrations path, then run the **AFIDs Validator Deploy** workflow against `enh/ai-tutor-learning-mode`. The Groq key is already configured; the tutor goes live automatically once the branch is deployed.

README.md

@@ -12,6 +12,31 @@ Anatomical fiducials (AFIDs) is an open framework for evaluating correspondence
 
 # [afids-validator (https://validator.afids.io)](https://validator.afids.io)
 
+## Guided Learning (AI tutor)
+
+The validator includes a guided-learning mode at `/learn` — an interactive tutor that walks users through placing anatomical fiducials and gives feedback on each placement. It is powered by any OpenAI-compatible language model.
+
+**Default model:** [Groq's Llama 3.3 70B](https://groq.com) (`llama-3.3-70b-versatile`) — a free, open-source model. When an API key is present, the endpoint and model default to Groq automatically, so the only thing you need to configure is the key.
+
+Configure via `.env` (see `.env.template`):
+
+- `LLM_API_KEY` — API key for the shared tutor. Get a free Groq key at https://console.groq.com/keys. **Setting only this is enough** — `LLM_BASE_URL` and `LLM_MODEL` default to Groq.
+- `LLM_BASE_URL` — OpenAI-compatible endpoint. Optional; defaults to Groq when a key is set.
+- `LLM_MODEL` — Model name. Optional; defaults to `llama-3.3-70b-versatile`.
+
+The tutor degrades gracefully:
+
+- Visitors can enter their own key/provider from the in-page **Settings** panel — their key stays in their browser and is never logged or persisted server-side.
+- If no key is configured anywhere, the tutor streams static reference material instead of failing.
+
+**Local, zero-key option:** run [Ollama](https://ollama.com) (`ollama pull llama3.2`) and leave `LLM_API_KEY` empty — the tutor talks to your local model, no key or cost.
+
+> RAG note: retrieval degrades gracefully to the built-in landmark reference when no embeddings store is present, so the tutor works without ingestion. Do **not** run `flask ingest-knowledge` against a Groq key — Groq serves chat models only, not embeddings.
+
+## Deployment
+
+Production is deployed via the **AFIDs Validator Deploy** GitHub Actions workflow (`.github/workflows/deploy.yml`, manually triggered), which builds a wheel and ships it to the server over SSH. Deployment configuration (including `PRODUCTION_LLM_API_KEY` for the shared tutor) is supplied through repository secrets. See `DEPLOY_HANDOFF.md` for the full step-by-step deploy runbook.
+
 ## Development
 
 `poetry` is used to manage dependencies. To install, run the following command:

afidsvalidator/__init__.py

@@ -2,6 +2,7 @@
 
 import os
 
+import click
 from flask import Flask
 from flask_migrate import Migrate
 
@@ -10,8 +11,10 @@
     ProductionConfig,
     TestingConfig,
 )
+from afidsvalidator.learn import learn
 from afidsvalidator.model import db, login_manager
 from afidsvalidator.orcid import orcid_blueprint
+from afidsvalidator.rag import KnowledgeChunk  # noqa: F401 — registers model
 from afidsvalidator.views import validator
 
 
@@ -59,6 +62,46 @@ def create_app():
     # Register blueprints
     app.register_blueprint(validator)
     app.register_blueprint(orcid_blueprint)
+    app.register_blueprint(learn)
+
+    # ── CLI commands ──────────────────────────────────────────────────────────
+    @app.cli.command("ingest-knowledge")
+    @click.option(
+        "--file",
+        "filepath",
+        default=None,
+        help="Path to a plain-text document to ingest (optional).",
+    )
+    @click.option(
+        "--source",
+        default=None,
+        help="Source label for the document (required with --file).",
+    )
+    def ingest_knowledge_cmd(filepath, source):
+        """Embed and store knowledge chunks in the RAG store.
+
+        Without arguments, ingests the 32 AFIDs landmark definitions from
+        landmark_info.py.  Pass --file and --source to ingest additional
+        documents such as the AFIDs protocol paper.
+
+        Example:
+            flask ingest-knowledge
+            flask ingest-knowledge --file afids_paper.txt --source afids_paper
+        """
+        from afidsvalidator.rag import ingest_landmarks, ingest_text_file
+
+        if filepath:
+            if not source:
+                raise click.UsageError(
+                    "--source is required when using --file"
+                )
+            click.echo(f"Ingesting {filepath} as source={source!r} …")
+            count = ingest_text_file(filepath, source)
+        else:
+            click.echo("Ingesting 32 AFIDs landmark definitions …")
+            count = ingest_landmarks()
+
+        click.echo(f"Done — {count} chunk(s) stored.")
 
     return app