What a Mail Server Checks Before It Accepts the Message

« Blog · By

The common picture of spam defence goes like this: the message arrives, a filter reads it, weighs subject and text and decides between inbox and spam folder. That exists, and it is the last step. By far the larger part of unwanted traffic is refused before a single line of the text has been transmitted.

The reason is plain: reading text costs processing time, looking up an IP address costs almost none. A server that accepts every connection to the end in order to judge it afterwards makes itself a target.

The conversation before the content

SMTP is an exchange with a fixed order. Every step is an opportunity to end the connection — and most connections end well before the last one.

1 · TCP connection 2 · greeting 3 · EHLO 4 · MAIL FROM 5 · RCPT TO 6 · DATA blocklist, connection rate, type of network speaking before the banner means not a mail server valid name? PTR present? does it resolve back? SPF, and: does the sender domain exist at all? does the recipient exist? greylisting only here does inspection of the content begin
Five checks sit in front of the first line of text. They cost the recipient almost nothing and the sender everything, if one of them is not set up properly.

Step 1: the address alone

Before anything is said, the origin of the connection is known. That is enough for three decisions:

  • Blocklists. One DNS query, answered in milliseconds. If the IP is on a list the operator treats as hard, this is where it ends.
  • Connection rate. How many connections came from this network in the last minute? Open too many too fast and you are throttled or refused.
  • Type of network. IP ranges known as consumer access networks are refused for inbound mail outright by many servers. There is no prejudice behind that, only the observation that almost nothing but compromised machines delivers directly from there.

Step 2: speaking too early

The server sends a greeting line beginning with 220. Only afterwards is it the client's turn. Some servers delay that line deliberately by a few seconds and check whether the client starts sending anyway.

A well-behaved mail server waits. Simple sending programs of the kind that run on compromised machines skip the wait, because they could be delivering elsewhere in that time. Speaking before the greeting is self-identification.

Step 3: EHLO and reverse resolution

With EHLO the client states its name. Three things are checked:

  1. Is the name a fully qualified domain name at all? An EHLO localhost or a bare IP address stands out immediately.
  2. Is there a PTR record for the IP? That is: does the IP address resolve backwards to a name? A missing record is grounds for refusal at many recipients.
  3. Does resolution agree in both directions? The name from the PTR must resolve forward to the same IP again. This is called forward-confirmed reverse DNS, and it is the only one of these checks that cannot simply be asserted: the PTR is set by the network operator, the A record by the domain holder. Controlling both means genuinely belonging to both.

If the client additionally claims to be the receiving server itself — EHLO mail.company.com to mail.company.com — the case is clear. A server does not call itself.

Steps 4 and 5: the addresses

At MAIL FROM the SPF check runs — it needs the IP and the envelope domain, and both are now available. In addition it is checked whether the sender domain exists at all and can receive mail. A domain with neither MX nor A record cannot take a reply and is therefore useless as a sender.

At RCPT TO it is decided whether the recipient exists. Here lies the difference between a refusal and a later bounce notification: a server answering 550 now refuses — the notification is generated by the sender. A server that accepts first and notices later that the address does not exist has to send a notification itself, often to a forged sender.

Greylisting applies here too: a provisional refusal with 4xx requiring a second delivery attempt after some minutes. Why the first message therefore sometimes arrives noticeably later is described separately. The number of refused recipients per connection is also counted: trying ten non-existent addresses in a row means working through a list.

What the error text gives away

Because each stage has its own error, the refusal text says fairly precisely what went wrong:

Message, in substanceStage that failed
Client host rejected / blocked using …1 — the IP is on a list
Improper use of SMTP command pipelining2 — spoke too early
Helo command rejected: need fully-qualified hostname3 — the EHLO name is unusable
cannot find your reverse hostname3 — PTR missing or not matching
Sender address rejected: Domain not found4 — the sender domain does not exist
Recipient address rejected: User unknown5 — the address does not exist
Greylisted, try again in …5 — provisional, not an error

The distinction between 4xx and 5xx matters more than the text: 4xx means "again later", 5xx means "never again". A sender that keeps delivering after a 5xx makes itself suspicious; one that does not retry a 4xx is not a proper mail server.

What follows for your own sending

The list is short and almost boring — which is exactly why it gets overlooked:

  • Have a PTR record set and make it resolve forward. The network operator enters it, not you. For self-hosted sending this is the most common cause of blanket refusals.
  • Set EHLO to the same name the PTR points to. Not the machine's hostname, not the domain in the message.
  • Do not deliver directly from a consumer access network. Server sitting at home or behind an ordinary line? Then send through a relay.
  • Keep recipient lists clean. Non-existent addresses show up as early as step 5, long before any content is weighed.

The content of a message decides its delivery only once everything before it is in order. It pays to look for faults in the order the server checks them.