What the frell are they doing to my email

[me_and]

For quite some time (at least months, maybe a year or so), I've been getting the occasional email where what should clearly be a "." in a link has become "..": http://www.dreamwidth..org and the like. I'd assumed it was just a reasonably common typo for a reason I couldn't quite fathom, and never wondered much about it.

Until today, where I had a brief exchange of emails with the people who'd sent one such duff link, and they reported that nobody else had had that problem. So I figured it may be something on my end, and dug a little deeper.

Have a look at this chunk of raw email content (edited to preserve style while removing private data):

2xEuthkrNGNbo&sa=3DD&usg=uehcAIUEQhRCSmhnsT_uAKXIUEhbhtahtdTH" styl=
e=3D"color:inherit;text-decoration:inherit"><br>http://www.indelicates.com/=
NotHere/IAmNotTheFileEither.txt</a></span></p><p style=3D"padding:0px;margi=
n:0px;font-size:11pt;font-family:Arial;height:11pt;direction:ltr"></p><p st=
yle=3D"padding:0px;margin:0px;font-size:11pt;font-family:Arial;direction:lt=
r">Both are zipped folders that contain the tracks, digital artwork and cre=
dits. If you are unsure which version to download then go for the Mp3s.<br>=
<br>This album has a story, which you can read here:<br><span style=3D"colo=
r:rgb(17,85,204);text-decoration:underline"><a href=3D"https://www.google.c=
om/url?q=3Dhttp://www.google.com/url?q%3Dhttp%253A%252F%252Fwww.indelicates=
..com%252Felevator%252F1.html%26amp;sa%3DD%26amp;sntz%3D1%26amp;usg%3DAFQjCN=
H19duvjmV80u8vgfp_dWO16zdz6w&amp;sa=3DD&amp;usg=3DAFQjCNGMpiSpcpd7bOcZlcorQ=
26jkoROUA" style=3D"color:inherit;text-decoration:inherit">http://www.indel=
icates.com/elevator/1.html</a></span></p><p style=3D"padding:0px;margin:0px=
;font-size:11pt;font-family:Arial;direction:ltr"><br>As ever, our PR budget=
 is paid in kind with the kindness of strangers. We believe that these thin=
gs are possible without gatekeepers and central authorities - but we are su=
stained in that belief only by your willingness to go along with them. As s=
uch, anything you can do to help spread the word about this crowdfunding ef=
fort will really, really help us out. Please share the link to:</p><p style=
=3D"padding:0px;margin:0px;font-size:11pt;font-family:Arial;height:11pt;dir=
ection:ltr"></p><p style=3D"padding:0px;margin:0px;font-size:11pt;font-fami=

This is HTML in a single dense block with line continuations marked with "=". As there's no reason not to, the lines are all the same length, a reasonably standard 76 characters. All the lines, that is, except one, which noticeably sticks out. And what's that at the start of the line? A double dot where there ought to be a single one. There are other examples in this email, while all the other dots are untouched.

It looks like somewhere in the various hops emails take on their way to my inbox, something is switching any "." in the first column of a line to "..".

Now begins the task of working out who runs that server and what the frell they're doing with my email (although I have suspicions as to the likely culprit). But. Wow. Good bug.

Comments

[davidgillon]

Now begins the task of working out who runs that server and what the frell they're doing with my email

And why!?! would be pretty prominent in my mind! I can understand processing the header, but the body? Even google-style data-stripmining shouldn't need you to rewrite the contents, you're just making work for yourself and inviting the bugs in.

[me_and]

I suspect the culprit is Sneakemail, which is a disposable email address service; they need to process the message content because I believe some of their features include rewriting in-body email addresses to use their obfuscated disposable addresses (so if I click on an email address it'll appear to come from the address they sent to rather than my private address), and automatically obfuscating message contents (so if I send a message and my real email address is quoted, it gets stripped). Granted the latter ought only happen on outgoing messages, and the former isn't a feature I have enabled, but I could believe they process all messages similarly.

[davidgillon]

Ah. That makes a certain amount of sense, and them the likely culprit!

[me_and]

Or, more likely, is that it's a completely mainline transformation that's only being performed in one direction, as simont points out…

[simont]

(This went past on my network page; apparently we both know kaberett. Hello.)

The phrase "dot stuffing" rang a vague bell in my head when I read this, and indeed, something relevant-looking is specified in RFC 5321 §4.5.2. To summarise: when mail is transferred by SMTP, sending a single "." on a line by itself is taken to terminate the message. Therefore, to make it possible to send emails one of whose actual intended lines of content consists of a "." by itself, the message is transformed during transmission to prefix an extra "." to any line starting with ".".

The idea, of course, is that the receiving SMTP agent undoes the transformation and the mail content is restored to its original state. But here, it looks very much as if some receiving SMTP agent along the chain has forgotten to undo the dot-stuffing.

[me_and]

(Hello! Yes! Also, for context, you added loop detection to Net at my indirect request :)

Ooh, good thought! I had wondered if it might be something like this from my hazy memories of interacting with SMTP servers using a Telnet client, but I'd assumed if there were anything special going on, it would only apply to lines that contained only ".", not any that started with a "." whether or not there were further characters.

[simont]

Ah, I remembered adding that to Net at someone's indirect request, but not whose. But also, now I look at that user icon rather than the one at the top of this post, I recognise you :-)

Certainly if you convert "." lines into "..", then you have to convert ".." lines in turn to something else, and that in turn and so on, otherwise you end up with some line contents which can either represent itself or be an escaped version of a different line, meaning the unstuffer doesn't know which one to turn it back into.

But yes, they could perfectly well have ruled that dot-stuffing applied to any line consisting of one or more dots and nothing else, rather than to any line starting with a dot even if a non-dot follows it. As long as the policy is consistent between all implementations, either (or many points in between) would work fine. I suppose they must have decided that mandating the very easiest of the options would give the best chance of nobody messing it up, and "dot-stuff any line starting with a dot" is plausibly that.

[xmc]

And if you don't nest conversion then you get the situation in From mangling where lines beginning with "From " sometimes get transformed into ">From " because mbox is a miserable format, and non-reversibly because ">From " doesn't become ">>From ", and so forth.

This is the email abyss, swallower of sysadmin souls.

[me_and]

Huh, yes. I remember pondering how that sort of escaping was done ~15 years ago when playing with SMTP using a telnet client, but I'd completely forgotten about it until now. Thank you for the reminder!

[kaberett]

<3!

[silveradept]

Rather impressive bug there. And excellent for noticing it - an extra dot wouldn't trip a lot of people's notice.