I heard that Exchange was purchased by Microsoft, but it seems clear that you were working on new development. Maybe it was just the MTA that was purchased? I also heard that Exchange was initally written in Pascal and was ported to C using a translator, and that's why it took so long to come out with the 5.0 version. Why was Exchange started with version 4.0? And who DID write that NAT unfriendly push notification (New Message Notification)? Why didn't they just utilize one of the two open TCP connections from the client to the Exchange server to send a push notification?
So what’s the deal with this whole push notification thingy anyway? Yeah, I know Joel says that nobody outside Microsoft starts sentences with "so", but I do. Basically push notification is the word we used for the feature in Exchange that allowed the server to tell the client that there was new mail available for the user (to be more accurate, it was a mechanism for the server to tell the client that "something happened on the server that you might be interested in").
Up until that point, all the existing email clients polled their server. Every minute, they’d send a request to the server saying "Hey, has anything happened?" There are two big problems with this. First, it means that the system doesn’t respond to realtime events - for example, if you started a search, the user wouldn’t see search results for tens of seconds, even though the server had search results within milliseconds. The second is that it’s bandwidth intensive.
When we do bandwidth analysis of this kind of problem, we need to make some assumptions about the number of email messages a person receives. For the sake of argument, let’s assume that each user receives 50 email messages a day, and sends 10 of them. So there will be 50 "interesting" events that occur during the course of an 8 hour business day. There are 480 minutes in 8 hours, so that’s one message every 10 minutes (since email traffic tends to be bursty, this isn’t 100% accurate, but it works). What this means is that 90% of the "Hey, has anything happened?" requests comes back "No."
Let’s assume the "Hey, has anything happened?" request takes up 100 bytes and one TSDU (a "transport service data unit", roughly a packet on the wire). The response "Nope, nothing happened" takes another 100 bytes and again one TSDU. This means that each client, sends 200 unnecessary bytes of traffic, and 2 TSDU’s every minute. Every hour, 10,800 unnecessary bytes and 108 TSDU’s are sent. That’s for each client. If you have 10,000 users in your organization (a medium size organization), this means that you’re sending 108 Megabytes of unnecessary data and 1.8 million extra TSDU’s each hour.
This is a big deal, especially back in 1994, when most networks were either 10MBps Ethernet or token ring. Also, many of our customers were running unroutable protocols like NetBEUI, which meant that they didn’t even have the option of segmenting their network to reduce traffic. Adding hundreds of megabytes to the bandwidth of a network could cripple the network. So we needed a mechanism to remove this load from the network.
The solution we came up with was "Push Notification". The idea was that the client would register that it was interested in notification with the server, then start listening for a response. When the server had something "interesting" to tell the client, it would send a ping to the client that said "Hey, something happened, you should look". When the client received this ping from the server, it would then issue the same "Hey, has anything happened?" message, and it’d get a list of the things that had changed.
There were a bunch of unexpected benefits that came from implementing push notification. For example, the user perception of the performance of searches became much faster. This is (roughly) because an Outlook/Capone search is done by creating a search folder on the server. The store on the server scans the database and adds the messages that match the search criteria to that search folder. And as each message is added to the folder, it generates a push notification to the client, which retrieves the current state of the folder. Without push notification, the client polled the server every 5 or 10 seconds, and the search results appeared in bursts, with push notification, the messages appeared in the search folder smoothly. There were several reviews of Exchange that gave Exchange’s search functionality high marks just because of this feature!
The reason we didn’t use the existing TCP connections was two-fold. First, and most important, not all of our clients ran TCP/IP. In fact, the majority of our clients for Exchange 4.0 ran either NetBEUI or IPX/SPX. On IPX, you didn’t even HAVE a connection, it was a datagram based RPC protocol. Second, the two connections were used for the address book provider (emsabpXX.dll) and the store provider (emsmdbXX.dll) for their RPC traffic to the server.
At the time, RPC barely supported asynchronous operations (it was possible but not on all platforms), and in order to use the existing connection, we would have had to totally rearchitect the communication protocol used between the store and the client. It was unclear if it would even be possible to support this on clients like MS-DOS or Windows. So using the existing store TCP connection wasn’t really an option for us at the time.