prompt change to be more explicit about handling race abbreviations - #181
Conversation
|
@muskankwalia Looks great! Just a few suggestions:
Also, can verify that these changes fix the pre-existing problem on the few samples that were having issues? (I.e., after you make the changes I suggest, can you run a quick test and verify that at least it fixes the problem as intended)? |
|
@chohlasa IMO it's fine to merge a prompt before verifying it is an improvement. that's the idea with the version suffix: we can merge a new candidate, evaluate it, and later pin it in the prod config if it turns out to be a successful improvement. doesn't harm anything to have unsuccessful candidates in the repo, and perhaps somewhat useful to have a record of them around. @muskankwalia the tests are failing due to formatting issues with the prompt. you can run |
|
@jnu Totally! I'm just trying to make sure we have some evidence this fix with address the narrow problem we're trying to address here, before running across a bunch of records. 100% agree we should wait until the full set of coverage evaluations are run before we roll it out to production, though, and that the next coverage evaluation might uncover problems that will require another edit before we deploy this set of revisions. |
chohlasa
left a comment
There was a problem hiding this comment.
@muskankwalia Looks good to me as long as you are able to test on a small set of docs that were having problems before and we see that it fixed those problems! Thanks for getting this together so quickly!
New language is on lines 2 and 9. Previous language around abbreviations on line 24 (line 22 in redact_20260421.txt) was cut.