Valid From header being rejected?

[tabascoterrier]

I've got a case where a From address header like =?UTF-8?B?Zm9vOmJhcg==?= <[email protected]> is being rejected by the headers plugin. It's an encoded-word version of foo:bar <[email protected]>, and it looks like the headers plugin is decoding the header, attempting to parse it, and fails with the unquoted colon.

const from_hdr = txn.header.get_decoded('From');
//...
addrs = addrparser.parse(from_hdr);

This pattern is also used in dkim_sign.

I suspect that the parsing should be done before any decoding, i.e. that example header should be accepted, but I can't find many sources to back that up so I'm curious as to what people think.

There's some more discussion on SO

[msimerson]

See also: jackbearheart/email-addresses#66 and jackbearheart/email-addresses#64

[tabascoterrier]

Thanks @msimerson. From my reading of RFC5322, the parse failure in jackbearheart/email-addresses#66 is the correct outcome - the colon in the display name, being part of the special characters, can't exist outside of a quoted string.

e.g. Something: something <[email protected]> is correctly rejected, but "Something: something" <[email protected]> (or Something":" something <[email protected]>, anything that quotes the special character) would be fine, and indeed they're happily accepted by address-rfc2822.

What I'm running into here is that in the headers plugin, the RFC2047 "encoded word" version of the display name is being decoded before the address is parsed, and I'm not sure that's correct. The only answer so far on SO quotes RFC2047:

NOTE: Decoding and display of encoded-words occurs after a structured field body is parsed into tokens. It is therefore possible to hide 'special' characters in encoded-words which, when displayed, will be indistinguishable from 'special' characters in the surrounding text. For this and other reasons, it is NOT generally possible to translate a message header containing 'encoded-word's to an unencoded form which can be parsed by an RFC 822 mail reader.

Which makes me think that the headers plugin should do the address parsing on the received header value rather than the decoded one, e.g.

const from_hdr = txn.header.get('From');
//...
addrs = addrparser.parse(from_hdr);

not

const from_hdr = txn.header.get_decoded('From');
//...
addrs = addrparser.parse(from_hdr);

Did you have any thoughts on that?

[msimerson]

This seems like a conversation that belongs over on the headers plugin page. In looking at the code in the headers plugin, the parsing of the From header is principally concerned with extracting the from domain. That necessarily needs to happen on the decoded value, or the headers plugin would need another dependency. I think.

One could just make a PR against the headers plugin and test before/after. But I have vague memories of switching that to get_decoded('From'), and I can't remember exactly why. It could be somewhere else in the code entirely. Ahah, haraka/Haraka#2537 is why we use get_decoded.