20 May 2012

Email with invalid headers, caught in spam filters, taken to be delivered

Submitted by Troy Rollo

In William Close Pty Ltd v City of Salisbury [2012] SAERDC 26, the South Australian Environmental Development and Resources Court found that an email that had invalid headers was properly delivered.

The email in this case included a "Content-Transfer-Encoding" header that was empty. This is clearly contrary to the relevant standard (RFC 2045 s6), and the behaviour of mail software seeing an unrecognised value for "Content-Transfer-Encoding" has been specified (RFC 2049 s2, paragraph (3)):

A mail user agent that is MIME-conformant MUST:

...

(3) Must treat any unrecognized Content-Transfer-Encoding as if it had a Content-Type of "application/octet-stream", regardless of whether or not the actual Content-Type is recognized.

The effect of this is that is would be contrary to the standards to recognise this email message as a valid email message. The rule says that when the Content-Transfer-Encoding is unrecognised, the content of the message should be treated as binary data, and the mail software is not entitled to look inside that data to try to extract some meaning from it.

That said, some software will ignore the rule and display the content as a text message.

In this case, the council receiving the email had subscribed to a spam filtering service which treated the invalid header as a sign of spam, quarantined the email and subsequently deleted it. From a technical perspective, this is very justifiable.

The problem though is that the spam filtering service's server was the designated system (in the MX record for the council's domain) for receiving email addressed to the council. Under the Electronic Transactions Acts in each State as in force at the time, that means that as soon as email is delivered to the designated system, it is taken to have been served on the recipient.

The Court then considered whether there was an implied requirement for the email to be legible. It found that there was, but decided that it was legible because some other recipients of the email were able to read it, and the experts stated it would be possible to read it.

While both these facts are true, it seems to miss the point. This is an email which, according to the standards, the receiving mail software should treat as a blob of binary data, not as a valid message. If conforming mail software is used to read it, it is for all practical purposes illegible. Recipients should not be expected to run mail software that does not conform to the standards, or to look at the underlying source of an apparently binary object to see if there might be legible text there.

Even if the email were legible, the Electronic Transactions Acts (both then and now) include an additional requirement:

at the time the information was given, it was reasonable to expect that the information would be readily accessible so as to be useable for subsequent reference

It hardly seems reasonable to expect information in an email will be accessible if it contains header defects that should cause conforming email software to treat the whole message as a bunch of binary data. It would also be unreasonable to expect information to be accessible if it contains the kind of defect that would cause spam filters to quarantine it.

The Court sidesteps this by saying that within the multipart email, the covering letter and attachment had valid Content-Transfer-Encoding headers, however the email client is not permitted to look at that, once it has established that the field at the email header level is invalid.

While I think the Court got it wrong in this case, this does raise a real risk with spam filters. A filter that just drops the spam without forwarding summaries of what it has done runs a real risk
(under the rule as then in force, and still in force in some States) that the receiving party will be treated as having seen an email that they never in fact saw.

There are some commercial spam filtering systems that get this right. In the open source arena, the only one I have seen that gets it right is Maia Mailguard.