E12 Developer Roadmap


In Exchange 12 (“E12”), we have focused our public API story on:

  • Web Services for accessing information stored on the server
  • Managed Agents for processing mail flowing through the server
  • Windows Scripting Shell (Monad) “cmdlets” for managing the server itself

The Sep 15 PDC presentation, Future Directions – Building Custom Applications for Exchange 12 APIs, provides a great overview of these great new developer interfaces.  Our new Web Services will offer remotely-accessible Outlook-compatible business logic for all PIM types. Our new Managed Agent APIs for processing mail flow provide full access to either the raw MIME or message parts Outlook renders. Our new Monad scripting experience delivers the power and ease-of-use that our customers deserve. All of these new API’s have been designed with Visual Studio development in mind.

 

We use these API’s ourselves to build layers of the E12 product (i.e. Outlook “12” calls the Web Services, E12 anti-spam is completely implemented as Managed Agents, and our new admin GUI is entirely coded to the new cmdlets). This first-hand experience with the new API’s gives me confidence that you’ll find it much easier when you write applications which interface with E12.

 

The introduction of the new web services has freed us to “de-emphasize” three of the APIs introduced in previous generations of Exchange. These de-emphasized API’s will be fully supported in E12 (i.e. existing applications will work unchanged), and will be supported for 10 years after E12 releases, based on our standard Microsoft Support Lifecycle. These APIs may not be in future releases of Exchange beyond E12. All new applications should be written using the new Exchange Web Services. The 3 “de-emphasized” APIs are:

The architectural improvements in E12 mean that some APIs can no longer be supported at all. Applications that use these APIs will not function correctly against E12:

 

API

Will need to be replaced by

ExWin32 (aka M: or IFS)

Web Services

EDK Gateway

Managed Agents

Transport Event Sinks

Managed Agents

CDOExM

Monad “cmdlets”

WMI

Monad “cmdlets”

CDOWF

Windows Workflow Foundation

Web Forms

ASP.net

 

For each of the transitions, we plan to provide great guidance and samples within the E12 SDK. I look forward to your feedback on these new APIs, and our SDK support for the transition.

 

Thanks,

 

Terry Myerson
General Manager, Exchange Server

Comments (31)
  1. Bobby says:

    What about MAPI stuff? Like the COM classes IMAPISession, IMAPITable, IMAPIFolder?

  2. John Merrill says:

    MAPI will be fully supported in Exchange 12.

  3. Simon Barratt says:

    I am becoming more wary of even considering integrating our applications with Exchange. Beyond the next release, you never know whether your application will continue to run withouht having to redo it.

    You have got to stop adding and dumping features with such regularity.

  4. Alexander Zammit says:

    Actually the presentation says that Event Sinks are going to be “de-emphasized”.

  5. John Merrill says:

    Good catch, Mr. Zammit. The PDC slides were incorrect, and Terry’s blog post was correct. I’ve submitted a corrected set of slides to be posted.

    Thanks for the help!

  6. Joe says:

    Things that popped into my head

    1. Dumping of WMI – Great, I hated the integration, it sucked. Especially hated the mailbox reconnect because of all of its issues. And the bugs in the Exchange_DSACCESS class making it nearly worthless for monitoring what DCs are really being used.

    2. What about management from older platforms and non-MS platforms? Say provisioning from UNIX/Linux? Not everyone is entirely MS and in fact the more MS has tried to push that way the more larger orgs have pulled away from it (think "avoiding pigeonholes"). The DS team has recognized this and is working towards considerable platform interconnectedness. Should be possible to mailbox enable a user without having to do it only from an MS machine with the latest and greatest tools. This is a very basic provision process.

    3. If MAPI is going to still be around and used, lets see updated Docs and books for it. Kind of sucks to have to go back to 5.5 docs for instance to figure out how to enumerate rules programmatically on Exchange 2003.

    4. KEEP IN MIND ENTERPRISE COMPANIES! We do things differently in many cases. We do not have a single domain all of the time. We have multiple domains and forests. A machine having to be a member of the forest that Exchange lives in is quite a hindrance on support especially if managing multiple Exchange Orgs either as a service provider or just because your large company for some reason has multiple orgs.

    5. I noticed in the slide deck the get-mailboxstatistics cmdlet, I assume that you can pass more params in if you only want some of the info and not all of it?

    6. Ditto on the mailbox create with new-mailbox, I didn’t notice where you specified server for the task, I assume you can or are you expecting sgdb to be a unique combination across a forest and across multiple forests if using exchange resource forests, or managing multiple forests, etc.

    7. Finally, if you are doing something that leverages something lower level, go directly from the level you are working at to the base level, don’t go down to one level and let it take you to the next and so on and so on. An example here is the .NET stuff for accessing AD in 1.0 and 1.1. Look at what is done for 2.0 and an interface that goes straight to LDAP and not through ADSI. This gets you away from worrying about bugs in each of those levels making your components worthless. I am a DS MVP and strong MS proponent but have no desire to use the .NET stuff from what I have seen from the DirectoryServices side. I am perfectly happy with LDAP and don’t have near the issues people do using ADSI and the current .NET DS stuff. Couple that with working with many of the largest companies in the world and knowing that provisioning is often not going to be from MS platforms makes me concerned by what I see here.

    8. Keep in mind that many people won’t or can’t jump on to E12 and its new tools right away, I have several Fortune 50 or bigger companies as customers who are stilling running Windows 2000 and Exchange 2000 and not looking to jump for at least a year. I just read in an article about MS turning 30 and SteveB and how he drives a 7 year old Ford to work everyday, it made me laugh. How would he feel if something broke and he was told, we can’t help you, you need to upgrade to Ford Car V2006 even though your current version does what you need. We would certainly do that if someone complained about how broken Windows 98 was working…

    joe

  7. Karl says:

    Comment on point 8 in previous post.

    I know for a fact that 3 of Europes largest telecom-actors (Mobile Providers) are in fact using Exchange 5.5 so before they are actually even considering upgrading to E12, well I suppose they will actually move on to Exchange 2K first, 2K3 then, and E12 by the time Microsoft exctually stop supporting E12.

    /Karl

  8. joep says:

    Do you happen to know what this means for Outlook. Will outlook start to use the WebService APIs?

  9. Vivek Sharma says:

    Re: Joe’s comment:
    http://blogs.technet.com/exchange/archive/2005/09/15/410941.aspx#411245

    Great to hear the feedback on WMI providers, please let us know what other areas have been problematic. On your other questions:

    #4: There are two items to discuss here. One is the ability to do cross-forest operations like moving mailboxes. For this we will provide cmdlets that accept credentials other than the current logon, enabling the scenario where you are working in Forest 1 but
    need to move data to Forest 2. For the second case, which is the ability to manage a remote machine, we do support remote operations and in some cases specifically providing the DC you want to operate on.

    #5: Actually the monad way to do it is to get all the data back and select only what you need. For example: $stats = get-mailboxstatistics "domainuser"; $stats.LastLogonTime. Or you can do get-mailboxstatistics | select-object LastLogonTime. Either method
    will work.

    #6: Yes your assumption is correct for the example given. You can actually pass in multiple things for that identity argument: GUID, DN, or a friendly string. All are acceptable.

    Thanks, ~vivek

  10. Joe says:

    Should I be scared that you didn’t respond to #2?

    :o)

  11. Vivek Sharma says:

    No you shouldn’t be scared :). The first thing to note is that vast majority of companies will find running their tools or scripts on an Exchange or Windows box to be sufficient. The second thing to note is that if there is a need to manage from a non-Windows box, there are several options:

    1) You can TS into an Exchange server (or a machine that has the tools installed on it) and run the scripts there

    2) You can use several toolsets to manage the AD directly. This is not what we recommend to our customers as then you have to do your own checks and ensure a working system when you modify settings in the AD directly, however, LDAP is a fine protocol that is accessible from many tools such as Perl, Python etc.

    3) You can run the windows version of the toolset (perl with Win32 extensions for example)

    4) You can use SFU to run your tools on a Windows server box to get the best of both worlds.

    5) You can write provisioning "front-ends" that live on a Exchange box but can be accessed through a web interface on a non-windows machine

    The last solution is one that many partners adopt, and is one that is accessible on a wide variety of platforms.

    Thanks, ~vivek

  12. EDDIE says:

    As with Transport Event Sinks replaced by Managed Agents, how about Store Event sinks?

  13. Chris says:

    Hi exchange Team,

    Just a minor question: Will it with E12 be possible to write plain text messages with the OWA-Client? Maybe a little bit OT but, I do not know where else I can post that request…

    thx

    Chris

  14. Joe says:

    Hi Vivek, of the options presented, the only method that actually sounds like something I would consider for managing from a non-MS platform is direct update of AD since it does allow non-MS technologies to natively work with it. However, as you say, "This is not what we recommend to our customers" and depending on where you look now, you find that it may not be a supported mechanism to manage Exchange at all.

    It seems silly for customers to have to write an interface to manage Exchange when you are in the process now of making changes to the management interfaces. It sounds like a good time to ask for and hopefully get, more flexibility in what you provide. At some point, you guys (Exchange Dev) need to realize that not everyone runs everything from Windows like many of the other groups are realizing. Look at the R2 offerings, how much of that is directly related to helping people interoperate amongst multiple platforms? You guys seem to be out of step with the rest of MS Dev. Also, it would suck for a customer to create an entire management interface only to have MS decide, well what we did in E12 was wrong, we are going to change it again. Most companies out there using your products are not software development houses.

    While on the topic of modifying AD directly, if you guys intend to continue using AD for the backend store of info, Exchange needs to be smarter about checking validity of info before using it. As mentioned, it is possible to directly modify things so that means that AD is an untrusted source of data. The data needs to be validated before being used. Duplicate values, etc in AD shouldn’t bunch Exchange up, it should run into it, validate it, and indicate that isn’t good and then ignore it until it has been corrected. Again, it is a data source that resides outside the body of Exchange so, according to all secure programming practices, be considered untrusted and needs to be treated as such.

    Also, Exchange/Outlook needs to either look at its own store or look at AD. This situation of looking in both places and doing things is a support quagmire. Permission and security is a great example, the mailbox ACL either needs to be in the store or it needs to be in AD, pick one. As well as how publicDelegates are handled. I previously ran into a great issue where Outlook would update the publicDelegates and the store piece would be cleared out but the AD Piece wouldn’t and Outlook would show you that it was all fine, right up until someone could still send mail on behalf of someone else and no one could figure it out until they looked at AD directly instead of with Outlook. There are probably many folks out there right now with multi-domain Exchange deployments who look at outlook and see publicdelegates and think that is right when in fact, AD has additional folks as publicDelegates on those accounts.

    Also, back to

    "#5: Actually the monad way to do it is to get all the data back and select only what you need. For example: $stats = get-mailboxstatistics "domainuser"; $stats.LastLogonTime. Or you can do get-mailboxstatistics | select-object LastLogonTime. Either method will work."

    Doesn’t that seem a bit heavy if say you are looking to get the size of mailboxes for all users across an org of 200,000 mailboxes or even 2000 or list which users have mailboxes over x MB? You have to pull all info for all users and then throw it all away except for one small piece of info for all users.

    It is a scaling issue. This stuff works fine in a lab with 2 exchange servers running with 50 mailboxes with unlimited network/cpu like I have in a home lab, but when you get to the real world the lack of imagination from MS in running large orgs comes to the forefront.

    This is evident when you look at the current tools and how they only allow you to work within a single Org or the WMI providers that don’t accept a domain and a userid or even the DNs being used since they point at people used to doing everything from the GUI which is generally not very efficient.

    I am afraid we are turning in some silly limitations for others. If you have to return all info for a mailbox to get one or two pieces is how you intend to do it, you might as well tell people now in good docs how to build their own management interfaces that run on the exchange servers so they can write their own efficient mechanisms for doing things.

    Again, what you have now even works fine in small companies, but MS is pushing to take over large companies and have huge email systems out there, I, being someone who works with companies on this stuff, would love to see it so I don’t have to sweat blood to make it work well, the way that MS shows the CEOs and CIOs in brief look how cool we are displays.

    Right now I am trying to figure out how to efficiently get a listing of all rules across all mailboxes in a 200k user org because of what appears to be a poor implementation of the rule "from a DL" which seems to have serious impact on Exchange servers because of the additional LDAP expansion queries that has to occur. You might think something like that wouldn’t be too difficult. You would be wrong. If I had only 50 mailboxes, I could easily just go look myself, that stops working somewhere around several hundreds of mailboxes. Heck, the "from a DL" rule wouldn’t even be an issue in a 50 mailbox org, the server could easily handle that additional expansion work again…. Again… scale issues and lack of imagination to see what can happen when you are dealing with a larger environment…

    I like MS products a lot. Exchange I don’t like quite so much. But my job right now is to work with it in large enterprises. I am an AD guy, love working on it and doing things with it. However, someone figured out I was really good at figuring issues out so they moved me to go work on Exchange, what does that say about Exchange? I think Exchange could be a great product, if I didn’t, I wouldn’t bother telling MS what I find wrong with it and I tell MS a lot of things I find wrong with it. My main wish for Exchange is to stop monkeying around with new stuff, get the core solid and stable and running well. The core being AD/Exchange integration, I still see Exchange as a perfect example of how not to write a AD enabled application as well as a great example on how not to implement security when using AD. Get that core straightened out, then lets start moving on to other things. In the meanwhile, lets also get management at scale handled as well. If we correct that AD/Exchange interface, that problem could almost solve itself since there is tools for management at scale for AD and/or lots of people with the current knowledge on how to do it through LDAP.

    joe

  15. Vivek Sharma says:

    Joe, we spend a lot of time working with customers (a large number of large, medium and small businesses) to gather their needs. In short, we have seen scenarios like what you have mentioned, and feel that we have addressed the vast majority of them. A large part of your response is supposition and speculation, which is completely expected as you haven’t see the product yet :). I think you bring up great points in your post. As Terry mentioned earlier and is alluded to in our slides, we will have betas in the future, which will be a great oppurtunity for you to test out your concerns.

    Thanks for your time, ~vivek

  16. Exchange says:

    Eddie: Store event sinks are the same as EXOLEDB, which is de-emphasized.

  17. Joe says:

    Thank you Vivek, seriously. It is good to see you guys out here blogging and taking the questions and responding to them.

    I just have years of frustration built up from working with Exchange and trying to make it work smoother or better in large environments (150k mailboxes+ mostly). I have been in the meetings where we were promised something would be all better in the next version and well, it might be, but not for any of the circumstances we had specifically described. The WMI mailbox reconnect was a perfect example. On the flip side I have been in meetings where a fix is finally promised and it actually gets delivered, the DSACCESS supplying GCs to clients based on the domain of the client is a perfect example there. It was extremely painful for me to fight for that and took months, but once you said you would fix it, it got fixed, it just came out in SP2 which is what was promised in 3/2004.

    I fully admit I haven’t seen the new version, I hope to see the docs for the SDK fairly soon so I can start sniffing out what I think could be the holes so they can be further tested. It is difficult to just test things in hopes that you find something that is going to really be painful in a 200k mailbox org. You just sort of have to think of the tasks and how they would be accomplished.

    I have high hopes for the new edition, I hope it is everything that is forecast and more. I do have a concerned spot though already in terms of how that info retrieval will work.

    I just can’t visualize pulling all of that data across the network from a loaded Exchange server and plucking out what you need at the client and it being very efficient when you need just some small piece of info for every mailbox. I would think of it like needing the whenChanged attribute of every object in a domain and having to retrieve every attribute and throwing them away and keeping just whenChanged. In a smallish environment of ~7400 objects you are talking ~2 seconds vs ~25 seconds and ~700KB vs ~17MB. A lot of waste there. Just think if every query to AD required you to return all info about every object you returned and it was up to Exchange to filter out what it really needed? I expect you guys would be jumping up and down saying this is insane, we can’t be performant in an environment like this!

    Anyway, I will be out in Redmond next week for the MVP summit, hopefully you guys will be putting on a dog and pony show showing this stuff off and not just to the Exchange MVPs. The DS MVPs mostly seem to be involved with Exchange in some way or another as well. I would be impressed to see 10-15 non-clustered Exchange 12 servers running out of EEC fully loaded up with say 5k users a machine with simulated load of about 3500-4000 users and then "quickly" pulling some piece of data about each mailbox from all mailboxes across the WAN. Not asking for much am I?

    take care, joe

  18. Laughing John says:

    First I’d just like to say thanks for this site, it’s really helpful and it’s nice to be able to communicate with you guys.

    I work for a software house who have specialised in Outlook/Exchange solutions, so naturally we’re very worried about the directions Exchange is taking.

    Our solutions are often based on public folders, so of course we’re very upset about the loss of those.

    What we’ve struggled with is the constant introduction of new ‘strategic’ directions and new APIs only to find that they then get replaced or become ‘legacy’.

    What we’re crying out for here is one API that lets us get to all of the features of Exchange, ideally from any box (client as well as server), is simple to program and performs well. A .Net based class library would be great.

    There’s so many unsatisfactory ways to access Exchange/Outlook its ridiculous.

    We’ve tended to stick with CDO 1.21 as that’s remained constant (perhaps a little too constant), is accessible from the server and the client and performs. Additionally we use OOM on the clients (swapping to CDO to get any decent performance and access things we can’t in the OOM). We also use sink events (although we don’t like them much).

    The loss of public folders makes a lot of it academic to us as we will probably have to migrate out of Exchange altogether.

    I broadly welcome the web services,

    I just hope that they and new ‘stated direction’ is are to stay and won’t go the same way as all of the others.

    Any news on what’s going to replace public folders and how we’re going to migrate ?

    Personally I’d like to have seen the direction as:

    1) Migrate the database to SQL server.

    2) Replace MAPI with an up to date API accessible from .Net.

    3) Keep and improve public folders and forms.

    4) Migrate the forms engine to Winforms.

    5) Migrate the VB script in Outlook forms to the new .net ‘scripting’ engine.

    6) Web services for access where the other API is not available.

    7) An improved workflow/eventing model on the Exchange box.

    A shame it’s not going to happen ….

  19. John C. Welch says:

    And those of us using Entourage, which requires DAV do?

    SIt around and watch as there are yet another set of Exchange features that we can’t use on a Microsoft Client?

    Putting DAV into "there but we don’t like it" mode and getting rid of public folders make moving past Exchange 2003 effectively impossible for those organizations that support their Mac users on Exchange.

    What’s being done to address those issue on the Exchange side?

  20. Andy Ruff says:

    John, the Entourage team meets regularly with the Exchange team to help us determine what direction we need to go in the future. For Entourage 12, we’ll continue to use DAV for our synchronization mailbox.

    I’ve seen and coded against the Exchange 12 web services. They are going to make developing Exchange solutions, such as Entourage, a whole heck of a lot easier. Our ability to adopt these changes will have to balanced against the Entourage team’s resources and the number of Exchange 2000, 2003, and 12 users we have to solve problems for. As you well know, we still don’t have everything that many people think is important, regardless of Exchange server version (e.g. OOF).

    I don’t think you should read so much doom and gloom into things. The Entourage team has worked hard to demonstrate we want to make things better with the recently released SP2. Many of the optimizations you see that resulted in improved synchs were the result of us working closely with the Exchange team. What makes you think that this is going to change?

  21. Andy Ruff says:

    John, the Entourage team meets regularly with the Exchange team to help us determine what direction we need to go in the future. For Entourage 12, we’ll continue to use DAV for our synchronization mailbox.

    I’ve seen and coded against the Exchange 12 web services. They are going to make developing Exchange solutions, such as Entourage, a whole heck of a lot easier. Our ability to adopt these changes will have to balanced against the Entourage team’s resources and the number of Exchange 2000, 2003, and 12 users we have to solve problems for. As you well know, we still don’t have everything that many people think is important, regardless of Exchange server version (e.g. OOF).

    I don’t think you should read so much doom and gloom into things. The Entourage team has worked hard to demonstrate we want to make things better with the recently released SP2. Many of the optimizations you see that resulted in improved synchs were the result of us working closely with the Exchange team. What makes you think that this is going to change?

  22. John C. Welch says:

    Well, now that someone from the Mac BU’s said something, i’m not as concerned. I didn’t think these changes would mean you guys got banned from talking to Exchange. I’m not reading doom and gloom, I’m asking for clarification before the noise on this gets higher.

    But, and you know a lot of this already, getting this kind of information out of y’all is sometimes a bit…tedious?

    However, since you’re going to be using DAV primarily in E’rage 12, then what does this rather confusing sentence mean:

    "The introduction of the new web services has freed us to “de-emphasize” three of the APIs introduced in previous generations of Exchange. These de-emphasized API’s will be fully supported in E12 (i.e. existing applications will work unchanged), and will be supported for 10 years after E12 releases, based on our standard Microsoft Support Lifecycle. These APIs may not be in future releases of Exchange beyond E12."

    Does this mean no more DAV in Exchange post – 12, or that it’s going to be there until 2015 regardless of version?

    How are the APIs that won’t work at all affect Entourage?

    What does "work unchanged" mean? Is it a freezing of capabilities of that API ("nothing new will be available to DAV in E12"), a limited set of features ("these new features will be available to DAV, but no others").

    Saying "support for ten years" and then in the same sentence saying, "these APIs may not be in future releases of Exchange beyond E12" is a tad confusing. Unless there’s not going to be a new version of Exchange for ten years after E12. That would then make sense.

    Since E’rage 12 is going to use DAV, how does web services benefit E’rage users?

    Is DAV going to still be a "main" access protocol ala MAPI, or is it going to be like IMAP and POP, and something that you have to do extra work to use? (Note: please keep in mind that a lot of Exchange administrators are, to old line email administrators, on the ignorant side, and if it takes more than the basic install and setup to use, they really fight using it. I had one offer to buy me an Apache box rather than go through the rather trivial work to enable multiple internet email aliases in Exchange 2000. Of course we only finished migrating off of Exchange 5.5 last year. Did I mention that Exchange administrators can be odd sometimes?)

    This isn’t theoretical…E’rage users are not just sitting in a design dept. somewhere. One of them is the #2 person at my company. if he’s not happy, the company’s not happy. (and yes, I’m pretty much ecstatic with SP2…he’s FINALLY, FINALLY going to drop that abomination that is Now Contact and Up To Date 5 with the improvements in SP2)

    But answering these questions now, in a clear manner will alleviate tons of problems when E12 comes out. It’s always easier to say "Here, read (link) when someone asks "What are the issues with E’rage 2004/12 and Exchange 12" than it is to try and guess based on what you can scrounge out of multiple exchange – related blogs. That in turn helps you avoid that glassy stare y’all get at Macworld when you hear the same exchange question for the humpty thousandth time.

    Oh yeah, and everyone doing Entourage supports owes you guys beer for the Entourage web log. Now a post or two about what E12’s changes mean there would get that upgraded to good beer ;-)

  23. John, let me take a stab at these.

    First, the 10-year window means that WebDAV will be supported in Exchange 12 from day 1 of its release to day 1+10 years. MS’ standard lifecycle support policy is to support products for a minimum of 10 years after release. When Exchange 13 ships, if it doesn’t include DAV suport, that won’t make E12’s DAV stop working.

    Second, I read "work unchanged" to mean that DAV applications will be able to access the same set of properties that they now get, with no new ones added. However, since I think you can get pretty much any DAV property if you know the MAPI proptag, this shouldn’t be an issue for Entourage.

    Third, Entourage doesn’t use any of the APIs that are being cut. (AAMOF, most of those APIs are pretty rarely used, which I suspect is one of the reasons they’re being cut.)

    Fourth, DAV is already a main protocol; you have to do something to turn it *off*. In Exchange 2003, IMAP and POP are disabled by default, but WebDAV is on by default because OWA depends on it. I don’t expect that to change in E12 (but I don’t know for sure).

  24. nickw says:

    Hey folks,

    Will E12 be available in both 64 and 32 bit versions?

  25. What’s the best avenue to get signed up for the beta? We’re a Certified Partner if that helps.

  26. Anonymous says:

    The Exchange Team has a great new post on the future of Public Folders in Exchange 12 and understanding…

  27. Anonymous says:

    I wanted to give you an intial heads up as to what to expect in E12 this post is just a very high level…

  28. Anonymous says:

    Slowly, but surely, I am getting caught up. Here’s the log from Part 6 of the 24 Part Exchange 2007 Webcast

  29. Anonymous says:

    A couple days ago I posted a link to upcoming Outlook Object Model changes and new features, I wanted

  30. Anonymous says:

    It’s late so I’m going to keep this brief but since we don’t have any public content I wanted to get

Comments are closed.