Recognize comments in the remaining addrparse states - #141
Open
gaoflow wants to merge 1 commit into
Open
Conversation
3e29988 added comment handling to the Unquoted and AfterBracketedAddr states and noted that "in general comment support is still lacking". RFC 5322 allows CFWS between any two tokens of an address, but the other states push '(' into whatever string they are accumulating, so a comment ends up in the addr or the display name: addrparse("(ab) x@y.com") -> addr "(ab) x@y.com" addrparse("\"Foo\" (c) <x@y.com>") -> display_name "Foo (c)" Comment bodies containing '<', ',', ':' or ';' were worse than that: they derailed the parse into a hard error. Handle '(' in Initial, AfterQuotedName and NameWithEncodedWord as well. The QuotedName state is deliberately left alone, since a paren inside a quoted-string is qtext (RFC 5322 3.2.4), as is BracketedAddr, whose contents this parser passes through verbatim.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
3e29988taughtaddrparseabout comments in theUnquotedandAfterBracketedAddrstates, and its message says "in general comment support is still lacking". This is the rest of that:(is a comment opener in 2 of the parser's 7 non-comment states, and in the other 5 it just gets pushed into whatever string the state is accumulating.RFC 5322 puts
[CFWS]on both sides ofatom,dot-atomandquoted-string, and §3.2.3 says outright that "the optional comments and FWS surrounding the rest of the characters are not part of the atom", so a comment is legal (and semantically absent) in all of those positions. The last two lines above are the sharper problem:ctextis everything printable except(,)and\, so<,,,:and;inside a comment are ordinary content, but the parser sees them in the address state and derails.This is the same defect as #65, one position over — your comment there was "the address is not bracketed so the parsing is wrong", which is the ruling I'm applying to the remaining states.
The change adds a
(arm toInitial,AfterQuotedNameandNameWithEncodedWord, reusing the existingcomment_returnmechanism. Three lines each, no new state.Deliberately not changed, and I checked each one rather than assuming:
QuotedName— a paren inside a quoted-string is qtext (§3.2.4), so"F(o)o" <x@y.com>must keep it. Asserted in the test.BracketedAddr—<(c)x@y.com>is technically CFWS, but this parser hands back the contents of<...>verbatim, whitespace and all (< x@y.com >gives" x@y.com "), so stripping comments there without also normalizing whitespace would be half a change. Your call; happy to do it in a follow-up if you want it.x@y.com ((a)b)givesx@y.com b)today and still does — theCommentstate has no depth counter, and adding one needs the quoted-pair handling that Unescape quoted-pairs inside quoted-strings #140 puts in that same match arm, so I left it for after Unescape quoted-pairs inside quoted-strings #140 lands rather than write it twice. One consequence worth flagging: a leading nested comment changes from(b) x@y.comtob) x@y.com. Both are wrong; it now fails the same way the trailing position already does instead of differently.Foo (c) Bar <x@y.com>yields"Foo Bar"with a doubled space. That is pre-existing — master does it for this already-supported position — so I pinned it in the test rather than quietly normalizing phrase whitespace.<"a>b"@host>) are still out of scope, same as in Unescape quoted-pairs inside quoted-strings #140.One correction to my own framing. I originally had this down as agreeing with both Python and Go. Go does not actually agree:
mail.ParseAddressList("(ab) x@y.com")returnsmail: missing word in phrase: mail: invalid string. Go'snet/maildoesn't accept a leading comment at all, and it refuses 33 of the 68 cases I tested, so it's not evidence either way for most of them. The oracle I actually used is Python'semail.headerregistryparser, cross-checked against the ABNF. On the addr sets it returns, mailparse goes from 26/68 to 51/68; the remaining 17 are the exclusions listed above.Testing.
cargo test --all— 54 → 56 unit tests, plus 25 doctests, green.cargo fmt -v -- --checkclean.cargo clippy --all-targetsreports the same 3 pre-existing warnings before and after.I mutation-tested the new tests in both directions, 8 mutants, all red, no survivors. The useful one: making
Commentreturn toInitialinstead of the state it came from reddens your ownreal_world_examples— both the Postfix(mail delivery system)row and the(GitHub Staff)row — which is the check that this doesn't quietly widen what gets thrown away.The second test goes through
parse_header+addrparse_headerbecause theNameWithEncodedWordstate can't be reached from plainaddrparse(there are noDecodedWordtokens), soFrom: =?UTF-8?B?Rm9v?= (c) <x@y.com>is the only way to exercise that arm.Branched off
master, independent of #140 — the two touch different parts of the function, and I'll rebase whichever lands second.