AI policy proposal

[lornajane]

Following on from our discussion and some of the recent TDC meetings, I've put together a starting point for an AI policy. We're agreed that AI should be permitted, but that there's an important point about the effort ratio between contributions and maintenance tasks.

• no schema changes are needed for this pull request

Claude proposed the outline of the doc after reviewing my preferred AI example policies, and also copyedited my document when finished.

[ralfhandl]

Thanks for preparing this!

[baywet]

The only one that I think we should allow here is the PR description.
I've been notoriously bad a hand writing PR descriptions, and thanks to your help, I believe I somewhat have improved.
However, lately I've been using AI to help me write better PR descriptions, based off the issue discussion, and the changes, and that's generally been much better than my poorly handwritten descriptions to set the state for other reviewers.

[miqui]

using AI to help me write better PR descriptions

Yeah, been doing this for quite some time. I got tired of writing of good descriptions because I was just going faster and faster. However, I ALWAYS reviewed the description before pushing.

[lornajane]

Perhaps the results are good with the tool in your hands, @baywet but that's not at all been my experience. AI-generated PR descriptions without expert input will at best describe information that is shown in the diff, and at worst both get that wrong and invent an incorrect explanation for what is changed and why.

[handrews]

@baywet I would also make a distinction between you writing a PR description with AI assistance to tell you if it reads well is distinct from someone generating a PR description by pointing AI at the code (which it may have written). idk if @lornajane sees a similar distinction, but I don't read your PR descriptions or comments and think "meh, AI wrote that."

[baywet]

Aaah so the context of this sentence is essentially "don't use AI to write those things, by itself", not "don't use AI at all"? That makes sense. Is there some way we could tweak it to reflect this more closely?

[lornajane]

I chose "generate" very intentionally. If the AI is generating the content that is shared: I don't think we want that. Those of you using the tools to augment what you want to say? I think we've already said elsewhere that sort of usage is not a problem.

[miqui]

@lornajane - well-crafted, pragmatic policy that strikes the right balance for a standards body.

The point about AI tools being trained on existing or outdated content is particularly astute for a standards project.

Just a thought:

No Assisted-by: git trailer convention. The broader open source ecosystem is converging on Assisted-by: as a standard commit metadata convention for AI-assisted contributions. Adopting it would make enforcement more tractable and machine-readable, especially for tooling.

Thanks @lornajane

[lornajane]

I did consider the additional metadata requirements, but I can't see us enforcing it so I didn't include it. It's just one more thing that most contributors will ignore and that we don't have the reviewer bandwidth to get hung up on - and I don't think it helps that much so it's not included. Thanks for prompting me to explain though!

[miqui]

using AI to help me write better PR descriptions

Yeah, been doing this for quite some time. I got tired of writing of good descriptions because I was just going faster and faster. However, I ALWAYS reviewed the description before pushing.