Skip to content

prompt change to be more explicit about handling race abbreviations - #181

Merged
jnu merged 4 commits into
mainfrom
prompt-refinement
Jul 8, 2026
Merged

prompt change to be more explicit about handling race abbreviations#181
jnu merged 4 commits into
mainfrom
prompt-refinement

Conversation

@muskankwalia

Copy link
Copy Markdown
Collaborator

New language is on lines 2 and 9. Previous language around abbreviations on line 24 (line 22 in redact_20260421.txt) was cut.

@muskankwalia
muskankwalia requested review from chohlasa and jnu July 8, 2026 14:39
@chohlasa

chohlasa commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

@muskankwalia Looks great! Just a few suggestions:

  • I'd move line 9 down to be a new paragraph down by line 24. I worry that prioritizing it so highly in the prompt may backfire and cause other issues.
  • In line 2 and line 9, should we be more precise and say "race/gender abbreviations like W/M", instead of just "race abbreviations"?
  • Can we use "gender" instead of "sex" in these additions?
  • On line 9, I'd swap out at least one of the examples for "B/M", and maybe also have one example with other formatting if you've seen any without a slash, e.g., "BM"

Also, can verify that these changes fix the pre-existing problem on the few samples that were having issues? (I.e., after you make the changes I suggest, can you run a quick test and verify that at least it fixes the problem as intended)?

@jnu

jnu commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

@chohlasa IMO it's fine to merge a prompt before verifying it is an improvement. that's the idea with the version suffix: we can merge a new candidate, evaluate it, and later pin it in the prod config if it turns out to be a successful improvement. doesn't harm anything to have unsuccessful candidates in the repo, and perhaps somewhat useful to have a record of them around.

@muskankwalia the tests are failing due to formatting issues with the prompt. you can run uv run pre-commit install in the repo to set up a tool that fixes these issues automatically when you make a commit.

@chohlasa

chohlasa commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

@jnu Totally! I'm just trying to make sure we have some evidence this fix with address the narrow problem we're trying to address here, before running across a bunch of records.

100% agree we should wait until the full set of coverage evaluations are run before we roll it out to production, though, and that the next coverage evaluation might uncover problems that will require another edit before we deploy this set of revisions.

@chohlasa chohlasa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@muskankwalia Looks good to me as long as you are able to test on a small set of docs that were having problems before and we see that it fixed those problems! Thanks for getting this together so quickly!

@jnu
jnu merged commit 5d35914 into main Jul 8, 2026
5 checks passed
@jnu
jnu deleted the prompt-refinement branch July 8, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants