Why TestConnectivity.Microsoft.com shows EOP as an open relay

The following article was written by Irol Melisa Pinto who is a Technical Advisor for Exchange Online Protection in Microsoft.

Hello EOP Admin’s out there! I am writing this article in the simplest form for a basic level of understanding. We recently worked with a couple of Tenant Admins concerned about the results seen in the SMTP test done through https://testconnectivity.microsoft.com (also known as the Microsoft Remote Connectivity Analyzer tool) and thought of just clearing some air around this topic through this blog.

The test in question here is the Inbound SMTP Email test and the results it returns when run against an Exchange Online mailbox.

The test results show the following.

So here I see a statement saying, “The open relay test message was delivered successfully to admin@testexchangeconnectivity.com.

Next I see a link saying “Tell me more about this issue and how to resolve it” which redirects to the article Open Relay Detected which states the following:

The Microsoft Exchange Analyzer tool attempts to send a test message using a recipient address that does not belong to the Exchange organization. If this operation succeeds, then the SMTP Server operates as an open relay and the following message is returned from the test.

"Open relay test message delivered successfully to Admin@TestExchangeConnectivity.com"

A computer that is running Microsoft Exchange 2000 Server, Exchange Server 2003 or Exchange Server 2007 can be configured as a mail relay, which is not recommended.

An Exchange computer that is configured as an open mail relay may be used to send unsolicited commercial e-mail, also known as spam. If other mail servers identify your Exchange computer as an unsolicited commercial e-mail server, then your Exchange computer may be added to block lists.

Oh boy! With this, the thought that immediately comes to our mind is – is EOP relay safe? How is EOP safe if it is now shown as an Open Relay which was not the case earlier? To understand further I did a little more research on this.

First, I did a domain look up for testexchangeconnectivity.com and it points to IP 157.56.138.170

Now if I check who owns this IP, I see that it is owned by Microsoft!
https://www.senderbase.org/lookup/?search_string=157.56.138.170

The DNS record for testexchangeconnectivity.com points to IP 157.56.138.170 which is Microsoft owned. We do know that testexchangeconnectivity.com is Microsoft owned public test tool. Don’t we know that already? :) So the above results were expected.

Next I checked this article Office 365 URLs and IP address ranges which includes 157.56.0.0/16 under Exchange Online IPv4 Addresses.

Next I used this tool - https://ipconvertertools.com/cidr2ipranges and I see the following:

The IP that testexchangeconnectivity.com used is actually a part of the O365 addresses.

Once again to quickly summarize our findings so far, the DNS record for testexchangeconnectivity.com points to 157.56.138.170 which is a Microsoft IP -> the same IP Address is also a part of Exchange Online pool of IP ranges.

The reason that the Remote Connectivity Analyzer SMTP test shows EOP as an Open Relay, is because it’s sending IP is trusted and seen as internal by EOP. This is why EOP accepts mail from this tool and will route it out to a third party.

As I am sure you know, EOP is not an open relay (and yes we Microsoft confirm that once again). EOP is an Auth-based relay. It follows a set of rules to confirm if an email can be accepted by EOP. Attribution happens in the envelope:

1. First is the Cert, if that matches an inbound connector, we accept the email.
2. Next is the IP, (from the EHLO) if that matches inbound connector, we accept the email.
3. Last is the RCPT TO: if that matches a domain in accepted domains, we accept the email.

In this case both the sending IP and the receiving IP both are part of Exchange Online IPv4 services and based on the above criteria mentioned, we really don’t need any further explanation. 

To reiterate, EOP is not an Open Relay. I do see that this Remote Connectivity Analyzer test has created a bit of confusion and we are working on fixing this incorrect test results.

Irol Melisa Pinto
Technical Advisor for Exchange Online Protection