Skip to content

Recognize comments in the remaining addrparse states - #141

Open
gaoflow wants to merge 1 commit into
staktrace:masterfrom
gaoflow:handle-comments-in-remaining-states
Open

Recognize comments in the remaining addrparse states#141
gaoflow wants to merge 1 commit into
staktrace:masterfrom
gaoflow:handle-comments-in-remaining-states

Conversation

@gaoflow

@gaoflow gaoflow commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

3e29988 taught addrparse about comments in the Unquoted and AfterBracketedAddr states, and its message says "in general comment support is still lacking". This is the rest of that: ( is a comment opener in 2 of the parser's 7 non-comment states, and in the other 5 it just gets pushed into whatever string the state is accumulating.

addrparse("(ab) x@y.com")          // addr           "(ab) x@y.com"
addrparse(r#""Foo" (c) <x@y.com>"#) // display_name  Some("Foo (c)")
addrparse(r#"(c) "Foo" <x@y.com>"#) // display_name  Some("(c) \"Foo\"")   <- quotes survive too
addrparse("(c) grp: x@y.com;")      // group_name    "(c) grp"
addrparse("(a<b) x@y.com")          // Err("Address string unexpectedly terminated")
addrparse("(a,b) x@y.com")          // Err("Invalid address found: must contain a '@' symbol")

RFC 5322 puts [CFWS] on both sides of atom, dot-atom and quoted-string, and §3.2.3 says outright that "the optional comments and FWS surrounding the rest of the characters are not part of the atom", so a comment is legal (and semantically absent) in all of those positions. The last two lines above are the sharper problem: ctext is everything printable except (, ) and \, so <, ,, : and ; inside a comment are ordinary content, but the parser sees them in the address state and derails.

This is the same defect as #65, one position over — your comment there was "the address is not bracketed so the parsing is wrong", which is the ruling I'm applying to the remaining states.

The change adds a ( arm to Initial, AfterQuotedName and NameWithEncodedWord, reusing the existing comment_return mechanism. Three lines each, no new state.

Deliberately not changed, and I checked each one rather than assuming:

  • QuotedName — a paren inside a quoted-string is qtext (§3.2.4), so "F(o)o" <x@y.com> must keep it. Asserted in the test.
  • BracketedAddr<(c)x@y.com> is technically CFWS, but this parser hands back the contents of <...> verbatim, whitespace and all (< x@y.com > gives " x@y.com "), so stripping comments there without also normalizing whitespace would be half a change. Your call; happy to do it in a follow-up if you want it.
  • Nested comments. x@y.com ((a)b) gives x@y.com b) today and still does — the Comment state has no depth counter, and adding one needs the quoted-pair handling that Unescape quoted-pairs inside quoted-strings #140 puts in that same match arm, so I left it for after Unescape quoted-pairs inside quoted-strings #140 lands rather than write it twice. One consequence worth flagging: a leading nested comment changes from (b) x@y.com to b) x@y.com. Both are wrong; it now fails the same way the trailing position already does instead of differently.
  • Foo (c) Bar <x@y.com> yields "Foo Bar" with a doubled space. That is pre-existing — master does it for this already-supported position — so I pinned it in the test rather than quietly normalizing phrase whitespace.
  • Quoted local-parts (<"a>b"@host>) are still out of scope, same as in Unescape quoted-pairs inside quoted-strings #140.

One correction to my own framing. I originally had this down as agreeing with both Python and Go. Go does not actually agree: mail.ParseAddressList("(ab) x@y.com") returns mail: missing word in phrase: mail: invalid string. Go's net/mail doesn't accept a leading comment at all, and it refuses 33 of the 68 cases I tested, so it's not evidence either way for most of them. The oracle I actually used is Python's email.headerregistry parser, cross-checked against the ABNF. On the addr sets it returns, mailparse goes from 26/68 to 51/68; the remaining 17 are the exclusions listed above.

Testing. cargo test --all — 54 → 56 unit tests, plus 25 doctests, green. cargo fmt -v -- --check clean. cargo clippy --all-targets reports the same 3 pre-existing warnings before and after.

I mutation-tested the new tests in both directions, 8 mutants, all red, no survivors. The useful one: making Comment return to Initial instead of the state it came from reddens your own real_world_examples — both the Postfix (mail delivery system) row and the (GitHub Staff) row — which is the check that this doesn't quietly widen what gets thrown away.

The second test goes through parse_header + addrparse_header because the NameWithEncodedWord state can't be reached from plain addrparse (there are no DecodedWord tokens), so From: =?UTF-8?B?Rm9v?= (c) <x@y.com> is the only way to exercise that arm.

Branched off master, independent of #140 — the two touch different parts of the function, and I'll rebase whichever lands second.

3e29988 added comment handling to the Unquoted and AfterBracketedAddr states
and noted that "in general comment support is still lacking". RFC 5322 allows
CFWS between any two tokens of an address, but the other states push '(' into
whatever string they are accumulating, so a comment ends up in the addr or the
display name:

    addrparse("(ab) x@y.com")        -> addr "(ab) x@y.com"
    addrparse("\"Foo\" (c) <x@y.com>") -> display_name "Foo (c)"

Comment bodies containing '<', ',', ':' or ';' were worse than that: they
derailed the parse into a hard error.

Handle '(' in Initial, AfterQuotedName and NameWithEncodedWord as well. The
QuotedName state is deliberately left alone, since a paren inside a
quoted-string is qtext (RFC 5322 3.2.4), as is BracketedAddr, whose contents
this parser passes through verbatim.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant