Add tiered steward review workflow#949
Open
JoaquinBN wants to merge 8 commits into
Open
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a three-tier steward hierarchy with AI-gated visibility, threshold-based escalation, appeal-aware AI processing, escalation lifecycle tracking, backend filters, reward adjustments, and tier-aware frontend queues and review controls. ChangesReview hierarchy
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
JoaquinBN
marked this pull request as ready for review
July 23, 2026 08:11
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/contributions/migrations/0084_enable_builder_review_hierarchy.py`:
- Around line 4-17: The migration functions enable_builder_review_hierarchy and
disable_builder_review_hierarchy should not be responsible for enforcing Builder
defaults on existing rows. Move the requires_ai_review and
escalation_threshold_points defaults into the ContributionType
creation/configuration path for Builder-category types, preserving them for
types created later. Make disable_builder_review_hierarchy non-destructive so
rollback does not overwrite later per-type configuration, and align the
implementation with the documented Builder-category default behavior.
In `@frontend/src/routes/StewardSubmissions.svelte`:
- Around line 441-445: Extract the repeated response handling in
handleToggleInteresting, handleContributionTypeUpdate, and handlePropose into a
shared helper that checks matchesActiveQueueFlags, removes mismatched
submissions via removeSubmissionFromPage, and otherwise performs the existing
in-place update. Reuse this helper from handleReview while preserving its
additional stateFilter condition.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 3812a0d0-f67b-452e-a8c4-32d3dcfc8082
📒 Files selected for processing (32)
CHANGELOG.mdbackend/CLAUDE.mdbackend/contributions/admin.pybackend/contributions/ai_review/views.pybackend/contributions/management/commands/review_submissions.pybackend/contributions/migrations/0083_review_hierarchy_fields.pybackend/contributions/migrations/0084_enable_builder_review_hierarchy.pybackend/contributions/models.pybackend/contributions/permissions.pybackend/contributions/serializers.pybackend/contributions/tests/test_ai_review_auth.pybackend/contributions/tests/test_escalation.pybackend/contributions/tests/test_review_submissions.pybackend/contributions/tests/test_state_transitions.pybackend/contributions/tests/test_steward_submission_search.pybackend/contributions/tests/test_tier_permissions.pybackend/contributions/views.pybackend/stewards/admin.pybackend/stewards/migrations/0013_steward_tier.pybackend/stewards/models.pybackend/submissions_review.mdbackend/users/serializers.pyfrontend/CLAUDE.mdfrontend/src/components/StewardSearchBar.sveltefrontend/src/components/StewardSubmissionCard.sveltefrontend/src/lib/searchParser.jsfrontend/src/lib/searchToParams.jsfrontend/src/routes/StewardSubmissions.sveltefrontend/src/tests/StewardSearchBar.test.jsfrontend/src/tests/StewardSubmissionCard.test.jsfrontend/src/tests/StewardSubmissions.test.jsfrontend/src/tests/searchParser.test.js
💤 Files with no reviewable changes (1)
- backend/contributions/ai_review/views.py
Comment on lines
+4
to
+17
| def enable_builder_review_hierarchy(apps, schema_editor): | ||
| ContributionType = apps.get_model('contributions', 'ContributionType') | ||
| ContributionType.objects.filter(category__slug='builder').update( | ||
| requires_ai_review=True, | ||
| escalation_threshold_points=400, | ||
| ) | ||
|
|
||
|
|
||
| def disable_builder_review_hierarchy(apps, schema_editor): | ||
| ContributionType = apps.get_model('contributions', 'ContributionType') | ||
| ContributionType.objects.filter(category__slug='builder').update( | ||
| requires_ai_review=False, | ||
| escalation_threshold_points=None, | ||
| ) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Do not make Builder hierarchy configuration a one-time data migration.
Line 6 configures only existing rows; Builder types created later retain False/None and bypass the intended AI gate and escalation. Line 14 also erases any later per-type configuration during rollback. Enforce the Builder default in the creation/configuration path and use a non-destructive reverse migration.
This conflicts with the documented Builder-category default behavior in backend/CLAUDE.md.
🧰 Tools
🪛 Ruff (0.15.21)
[warning] 4-4: Unused function argument: schema_editor
(ARG001)
[warning] 12-12: Unused function argument: schema_editor
(ARG001)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/contributions/migrations/0084_enable_builder_review_hierarchy.py`
around lines 4 - 17, The migration functions enable_builder_review_hierarchy and
disable_builder_review_hierarchy should not be responsible for enforcing Builder
defaults on existing rows. Move the requires_ai_review and
escalation_threshold_points defaults into the ContributionType
creation/configuration path for Builder-category types, preserving them for
types created later. Make disable_builder_review_hierarchy non-destructive so
rollback does not overwrite later per-type configuration, and align the
implementation with the documented Builder-category default behavior.
JoaquinBN
force-pushed
the
JoaquinBN/steward-review-layers
branch
from
July 23, 2026 08:57
3870039 to
5b58433
Compare
…ew-layers # Conflicts: # CHANGELOG.md
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
frontend/src/components/StewardSubmissionCard.svelte (1)
628-643: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftUse the submission-date multiplier for escalation previews.
getFinalPoints()uses the active multiplier map, but escalation is defined against the multiplier effective on the submission’s contribution date. A multiplier change can make the tier-1 UI incorrectly present an acceptance as final or escalated. Pass a server-calculated escalation result or a per-submission effective multiplier to this card.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/src/components/StewardSubmissionCard.svelte` around lines 628 - 643, Update the escalation preview flow around getFinalPoints() and acceptWillEscalate() to use the multiplier effective on the submission’s contribution date, not the current active multipliers map. Prefer consuming a server-calculated escalation result; otherwise provide and apply a per-submission effective multiplier when calculating final points, while preserving the existing threshold and tier checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/contributions/management/commands/review_submissions.py`:
- Around line 332-341: Move the submission appeal handling around the
`has_appeal` branch ahead of the global `if not templates: return` preflight, so
appealed submissions are counted, logged, and passed to `_mark_gate_reviewed`
during non-dry runs even when no templates are available. Keep
template-dependent rejection processing after this appeal path.
In `@backend/contributions/models.py`:
- Around line 84-96: Replace the __init__-kwargs-based tracking of
_explicit_review_fields_on_create with logic that identifies Admin-assigned
builder review values after form data is applied but before
ContributionType.save() mutates defaults. Update the related save/default logic
to preserve explicit False/None and excluded-field behavior, and remove reliance
on explicit_review_fields captured in __init__.
In `@backend/contributions/views.py`:
- Around line 2518-2526: Update the GlobalLeaderboardMultiplier.DoesNotExist
branch in the escalation check around effective_steward_tier and
escalation_threshold_points so a missing date-specific multiplier cannot bypass
escalation; conservatively require escalation or use a multiplier of 1 for the
threshold comparison, while preserving existing behavior when a multiplier is
found.
---
Outside diff comments:
In `@frontend/src/components/StewardSubmissionCard.svelte`:
- Around line 628-643: Update the escalation preview flow around
getFinalPoints() and acceptWillEscalate() to use the multiplier effective on the
submission’s contribution date, not the current active multipliers map. Prefer
consuming a server-calculated escalation result; otherwise provide and apply a
per-submission effective multiplier when calculating final points, while
preserving the existing threshold and tier checks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: af2508d8-a2bc-4bc0-a94a-7071cf5d4930
📒 Files selected for processing (36)
CHANGELOG.mdbackend/CLAUDE.mdbackend/contributions/admin.pybackend/contributions/ai_review/views.pybackend/contributions/management/commands/review_submissions.pybackend/contributions/migrations/0083_review_hierarchy_fields.pybackend/contributions/migrations/0084_enable_builder_review_hierarchy.pybackend/contributions/models.pybackend/contributions/permissions.pybackend/contributions/serializers.pybackend/contributions/tests/test_ai_review_auth.pybackend/contributions/tests/test_calibration_data.pybackend/contributions/tests/test_escalation.pybackend/contributions/tests/test_project_milestone_rubric.pybackend/contributions/tests/test_projects_and_milestones.pybackend/contributions/tests/test_review_submissions.pybackend/contributions/tests/test_reviewer_rewards.pybackend/contributions/tests/test_state_transitions.pybackend/contributions/tests/test_steward_submission_search.pybackend/contributions/tests/test_tier_permissions.pybackend/contributions/views.pybackend/stewards/admin.pybackend/stewards/migrations/0013_steward_tier.pybackend/stewards/models.pybackend/submissions_review.mdbackend/users/serializers.pyfrontend/CLAUDE.mdfrontend/src/components/StewardSearchBar.sveltefrontend/src/components/StewardSubmissionCard.sveltefrontend/src/lib/searchParser.jsfrontend/src/lib/searchToParams.jsfrontend/src/routes/StewardSubmissions.sveltefrontend/src/tests/StewardSearchBar.test.jsfrontend/src/tests/StewardSubmissionCard.test.jsfrontend/src/tests/StewardSubmissions.test.jsfrontend/src/tests/searchParser.test.js
💤 Files with no reviewable changes (1)
- backend/contributions/ai_review/views.py
Comment on lines
+2518
to
+2526
| threshold = contribution_type.escalation_threshold_points | ||
| if effective_steward_tier(request.user) < 2 and threshold is not None: | ||
| try: | ||
| _, multiplier = GlobalLeaderboardMultiplier.get_active_for_type( | ||
| contribution_type, | ||
| at_date=submission.contribution_date, | ||
| ) | ||
| except GlobalLeaderboardMultiplier.DoesNotExist: | ||
| pass |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Find how frozen_global_points is computed on Contribution creation to compare
# with the escalation final_points formula, and whether missing multiplier is handled.
ast-grep outline backend/leaderboard/models.py --items all --match 'Contribution|GlobalLeaderboardMultiplier' 2>/dev/null
rg -nP -C3 '\bfrozen_global_points\b|multiplier_at_creation' backend/contributions/models.py backend/leaderboard/models.pyRepository: genlayer-foundation/points
Length of output: 18794
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== views outline around escalation =="
ast-grep outline backend/contributions/views.py --match 'escalation|final_points|final_review' 2>/dev/null | sed -n '1,220p' || true
echo
echo "== relevant views lines 2460-2550 =="
cat -n backend/contributions/views.py | sed -n '2460,2550p'
echo
echo "== relevant models lines 440-486 =="
cat -n backend/contributions/models.py | sed -n '440,486p'
echo
echo "== GlobalLeaderboardMultiplier methods =="
cat -n backend/leaderboard/models.py | sed -n '292,345p'
echo
echo "== search escalation_threshold_points usages =="
rg -n -C3 'escalation_threshold_points|final_points|FinalPoints|GlobalLeaderboardMultiplier\.get_active_for_type' backendRepository: genlayer-foundation/points
Length of output: 50382
Conservatively gate escalation when the date-specific multiplier is unavailable.
A missing GlobalLeaderboardMultiplier falls through pass, so a tier‑1 steward can accept an award that is normally above escalation_threshold_points without creating the tiered-approval proposal. Escalation is an authorization control; handle the missing multiplier conservatively, e.g. by requiring escalation or using multiplier 1 for the threshold comparison.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/contributions/views.py` around lines 2518 - 2526, Update the
GlobalLeaderboardMultiplier.DoesNotExist branch in the escalation check around
effective_steward_tier and escalation_threshold_points so a missing
date-specific multiplier cannot bypass escalation; conservatively require
escalation or use a multiplier of 1 for the threshold comparison, while
preserving existing behavior when a multiplier is found.
Source: Path instructions
Threshold-enabled review decisions now grant deduplicated rewards to tier-one stewards, while escalated standard and rubric proposals share the existing accuracy-based economy. Proposal rewards scale down for point disagreements, and decision notes keep direct and proposer rewards separate. ## Claude Implementation Notes - backend/contributions/reviewer_rewards.py: Add proportional point slashing, shared reward grants, and deduplicated decision rewards. - backend/contributions/views.py: Record every escalation proposal and integrate proposal and direct-decision rewards. - backend/contributions/tests/test_escalation.py: Cover escalation outcomes, exclusions, deduplication, and combined rewards. - backend/contributions/tests/test_reviewer_rewards.py: Cover reward math, point adjustments, and persisted reward metadata. - backend/submissions_review.md: Document the expanded reviewer economy. - backend/CLAUDE.md: Record reward rules and deduplication behavior.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/contributions/reviewer_rewards.py`:
- Around line 46-54: Update the proportional-scaling condition in the reviewer
reward calculation to recognize proposed_points == 0 as provided, while avoiding
division by zero by handling the zero-proposal case explicitly. Preserve the
existing scaling behavior for nonzero proposed_points and final_points values,
and add a regression test alongside test_points_proportional_slash covering
proposed_points=0 with a nonzero final_points.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 2480f171-272a-457a-b3df-43c7dfcdeba1
📒 Files selected for processing (7)
CHANGELOG.mdbackend/CLAUDE.mdbackend/contributions/reviewer_rewards.pybackend/contributions/tests/test_escalation.pybackend/contributions/tests/test_reviewer_rewards.pybackend/contributions/views.pybackend/submissions_review.md
Reviewer reward scaling now treats zero as a provided point proposal without dividing by zero. Matching zero-point decisions retain the computed reward, while nonzero final awards reduce it to zero. ## Claude Implementation Notes - backend/contributions/reviewer_rewards.py: Handle zero-point proposals explicitly during proportional scaling. - backend/contributions/tests/test_reviewer_rewards.py: Add regression coverage for zero-to-nonzero point changes.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
backend/contributions/reviewer_rewards.py (1)
130-150: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winMake decision-reward deduplication atomic.
The duplicate check runs before
_grant_reward_contribution()acquires its lock. Concurrent requests can both observe no existing contribution and create duplicate rewards for the same user, submission, and action. Move the check under the sameselect_for_update()transaction, or enforce a database-level idempotency constraint and handle conflicts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/contributions/reviewer_rewards.py` around lines 130 - 150, Make deduplication in the decision-reward flow atomic by moving the existing contribution lookup for user, contribution_type, and notes into the same select_for_update() transaction used by _grant_reward_contribution, or enforce and handle an equivalent database idempotency constraint. Preserve the existing 'duplicate' return behavior while preventing concurrent requests from creating multiple rewards for the same user, submission, and action.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/contributions/tests/test_reviewer_rewards.py`:
- Around line 485-492: Add a second assertion alongside the existing
compute_reviewer_reward exact-match test, using proposed_points=0 and
final_points=0, and verify that it returns the base reward. Keep the current
nonzero-final assertion unchanged.
---
Outside diff comments:
In `@backend/contributions/reviewer_rewards.py`:
- Around line 130-150: Make deduplication in the decision-reward flow atomic by
moving the existing contribution lookup for user, contribution_type, and notes
into the same select_for_update() transaction used by
_grant_reward_contribution, or enforce and handle an equivalent database
idempotency constraint. Preserve the existing 'duplicate' return behavior while
preventing concurrent requests from creating multiple rewards for the same user,
submission, and action.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e2c04260-6ba0-412e-9513-84cef8e12cff
📒 Files selected for processing (2)
backend/contributions/reviewer_rewards.pybackend/contributions/tests/test_reviewer_rewards.py
Comment on lines
+485
to
+492
| self.assertEqual( | ||
| compute_reviewer_reward( | ||
| **exact_match, | ||
| proposed_points=0, | ||
| final_points=10, | ||
| ), | ||
| 0, | ||
| ) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Add coverage for zero-point exact matches.
The implementation intentionally preserves the computed reward when both proposed_points and final_points are 0, but this test only covers the nonzero-final case. Add an assertion for proposed_points=0, final_points=0 expecting the base reward.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/contributions/tests/test_reviewer_rewards.py` around lines 485 - 492,
Add a second assertion alongside the existing compute_reviewer_reward
exact-match test, using proposed_points=0 and final_points=0, and verify that it
returns the base reward. Keep the current nonzero-final assertion unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Review safeguards
Validation
The repository-wide frontend test command still reports existing failures in unrelated Profile, Dashboard, route metadata, and AuthButton mock suites; all steward-related frontend tests pass.
Deployment note
Migration
0084_enable_builder_review_hierarchyenables AI gating for every builder-category contribution type. Confirm external AI coverage for each builder type before deployment, or selectively disablerequires_ai_reviewin admin for uncovered types.Summary by CodeRabbit
is:escalated/not:escalated, new escalation-aware UI labels, and steward/user fields (steward_tier,escalated_at).