Server Solver

An RCP over HTTP Hurdle with Exchange

Admin encounters a showstopper as he tries to implement RCP over HTTP on an Exchange server.

• By Sekou Page
• 12/12/2005

Figure 1. RCP over HTTP error. (Click image to view larger version.)

Here's a description of my current system:

• Firewall: A dedicated computer running a Linux firewall controlling all the inbound and outbound connections
• Front-end: Windows Server 2003 Enterprise Edition SP1, Exchange Server 2003 SP1, RPC Proxy Server, Outlook Web Access, Web Server
• Back-end: Windows Server 2003 Enterprise Edition SP1, Exchange Server 2003 SP1, Domain Controller, Global Catalogue
• Outlook client: Windows XP Pro SP2, Outlook 2003

Any help here would be much appreciated.
– Andy Sim

RPC over HTTP (or RPC over HTTPS) allows your remote Outlook 2003 users on Windows XP Professional with Service Pack 2 to connect to Exchange 2003 server directly over the Internet without the use of a VPN. This means you can provide your users with a full Outlook experience. That is, when working remotely, they're no longer limited to the Outlook for Web Access client version. More importantly, there's a huge security benefit in that you can restrict VPN access to those who really need it – instead of those who simply require “full” access to email. For complete documentation on RPC over HTTP, see “Exchange 2003 RPC over HTTP Deployment Scenarios” on TechNet.

You’ll need to make sure you’ve met a few requirements before you try and deploy RPC over HTTP in your environment. Don’t take them lightly because if you fail to meet any of these, you’ll certainly have a tough time getting it to work properly. Basically, you need to be running the latest and greatest Microsoft products everywhere if you want to get this working:

#1: Windows 2003 Everywhere
It's important to remember that all servers that will be involved with your RPC over HTTP deployment should run Windows 2003

Exchange 2003 on Windows 2003: It’s probably pretty obvious, but the Exchange servers your user data resides on should run on Windows 2003 … this includes Exchange mailbox and public folder servers. Make sure you don’t forget about your public folder servers here, because even if your users or organization “never really use” public folders, their mailbox certainly does. Since their free/busy information is stored in public system folders, it will need to be accessible via RPC over HTTP as well if you want features like calendaring to work properly. In addition, if you have a front-end/back-end topology deployed, then the front-end servers that your users will access also need to run on Windows 2003.

Domain Controllers on Windows 2003: Yes, I know technically it is only global catalog servers that require Windows 2003 (I’ve heard both). It's certainly possible to get things working just fine by just making sure that all global catalog servers that Outlook 2003 clients and Exchange servers will use run Windows 2003. However, realistically there are often situations (especially in larger deployments) where you’ll find that attempting to selective upgrade your Active Directory global catalog servers to support RPC over HTTP creates more trouble than you'd expect.

First of all, Exchange tends to use global catalog servers as it sees fit and as designated in your Active Directory site topology. Most organizations will have redundant global catalog servers available to Exchange. So, it is possible that Exchange may fail over to another global catalog. Perhaps it will fail over to one that you haven't upgraded. The same goes for Outlook 2003 clients. Users that are configured to use RPC over HTTP when away from the office remain configured for RPC over HTTP when they return to the office LAN. There are situations where users will tell you their mail isn't working or their address book appears broken when they are in certain locations. For example a user visiting a remote office where you haven't upgraded your global catalog servers ends up having problems since the global catalog server in their local site is not running on Windows 2003.

The Hard(code) Road: Of course, you can get around all of this by hard-coding everything. You can hard-code your Exchange servers to use specific global catalog servers by manually configuring DSAccess, the internal process Exchange uses to communicate with and store directory information. You can hard-code Outlook users to use only specific global catalog servers. You can begin by knowing which global catalog servers your clients are using as in Knowledge Base article 317209, “How to Identify your Global Catalog Server Using Outlook 2000 and Outlook 2002.” You can then choose to specify a global catalog server running Windows 2003 by using KB 319206, “How to configure Outlook to a specific global catalog server or to the closest global catalog server.” You can configure your Exchange servers to use specific global catalog servers on the DSAacess tab of the server’s properties in the Exchange System Manager. For more information on controlling how Exchange interacts with DSAccess, see KB 250570, “Directory service server detection and DSAccess usage.”

Hard-coding everything becomes a difficult task to implement and manage, especially in larger environments. That being said, we all know doing things like this eventually comes back to haunt us. It is for this reason that I recommend … if possible … upgrading all domain controllers to Windows 2003. By doing so, you can then convert to Native Mode. Switching the domain to Native Mode will prevent anyone from introducing any Windows 2000 Domain Controllers (and global catalog servers) back into the domain and make your RPC over HTTP deployment that much easier.

Again, it's only required that global catalog servers that Exchange and Outlook users rely be running Windows 2003. However, a native 2003 environment eliminates any issues that may arise from this requirement completely. If you can’t upgrade all Domain Controllers to 2003, at least upgrade all global catalog servers. When a client makes an RPC request over HTTP, it is important to remember that this request is not changed when it is passed from one global catalog to the next. So this means that even if a client does not contact a global catalog directly, RPC over HTTP requests (which use an updated RPC) need to be understood by all global catalog servers the request is passed to.

If you don’t want to do this, and still require 2000 global catalog servers, it might be a good idea to attempt to optimize your global catalog configuration by controlling how they interact with the environment. This process is described in KB 306602, “How to Optimize the Location of a Domain Controller or Global Catalog That Resides Outside of a Client’s Site.”

Again, any server that your users or their Exchange servers or Outlook 2003 users will “touch” should be running Windows 2003.

#2: Exchange 2003 everywhere
This one should be obvious, so I’ll keep this short. Again, yes I know you can do this with a mixed environment. You can get away with not having all of your Exchange servers running 2003. Just like above, any Exchange server accessed by an RPC over HTTP user should run Exchange 2003 Standard or Enterprise. The mailbox server, the associated public folder server and any front-end RPC over HTTP servers must be running Exchange 2003 for RPC over HTTP to work properly. Keep in mind that these servers must be running on Windows 2003 as mentioned above.

In addition, consider scenarios where users will access other user’s mailboxes within their Outlook profiles. For example, if a secretary has his/her boss’s calendar in their Outlook profile, both the secretary’s and boss’s mailbox will need to be on Exchange 2003 servers. This is true even though the boss may never use RPC over HTTP. The mailboxes can certainly be different servers (when a front-end server is deployed), but both mailbox servers will definitely need to be on Exchange 2003 for RPC over HTTP access to work for the secretary, since the boss’s mailbox needs to know how to communicate properly with the front-end server (in this case). Considerations like this often make it easier to make sure all Exchange servers are running 2003 to avoid issues.

#3: Windows XP with SP2 and Outlook 2003
The client side requirements are fairly straightforward. Run Windows XP with Service Pack 2 (or Windows XP SP1 with hotfix 331320) and Outlook 2003. RPC over HTTP also works with clients running Outlook 2003 on Windows 2003 server if you are so inclined. In general, make sure your client machines are patched and up to date.

HTTPS Recommendation
Although it is not required, I do recommend that you do all of this over SSL, so that your traffic is encrypted and as secure as possible. Setting up RPC to work over HTTPS is actually not too difficult, but there are a few things to remember when deploying this if you want it to work properly.

The first test is to see if you can get Outlook Web Access working over HTTPS (SSL) for your server. If you can do this, you’re half way there. One of the most important things to remember when deploying RPC over HTTPS is that the certificate must be trusted on the client machines. There are plenty of administrators that deploy internal certificates (which are fine) and then tell users who use computers at home (non-domain members) to ignore the certificate warning when using Outlook Web Access. Well, RPC over HTTPS gives no warning and will simply fail if the certificate is not trusted. This means that you either need to have users with computers that don’t trust the internal Certificate Authority (CA) import your company’s CA or that you should use a certificate from a trusted external entity like GeoTrust or Verisign.

Attempt having your users authenticate to https://host.company.com/rpc/rpcproxy.dll to verify that they can access RPC over HTTPS without any issues or warnings. Note that users will see an “access denied” web page, which is expected. This page is not expected to display anything and is used only for testing the connection. You’ll want to verify what the security is set to for the RPC virtual directory as well if you have issues.

Another important thing to note is that the common name of your certificate must match what users enter in their Outlook configuration. This means you cannot use a wildcard certificate on your front-end server. A wildcard certificate is one that you purchase as “*.company.com” and allows you to specify any hostname for secured connections. Currently RPC over HTTPS requires that the certificate common name match exactly with what you enter in the Outlook configuration. If you purchased a certificate called “secure.company.com” then that's exactly what you need to enter into Outlook. You can have the user verify the certificate name by double-clicking the lock icon in the user’s browser or by checking in IIS on the web site’s Directory Security tab under “View Certificate”. So, if your user can connect to https://host.company.com/rpc/rpcproxy.dll without any certificate prompts or warnings and the certificate in use is not a wildcard certificate, then they will likely be able to connect using RPC over HTTPS. Using Basic authentication is recommended over NTLM in your Outlook configuration for compatibility with various firewalls.

Troubleshooting
When dealing with issues accessing RPC over HTTP or HTTPS it is important to make sure that you’ve really met all of the above requirements before going any further. For your specific problem, Andy, I’d recommend that you first test connecting on your LAN and without cached mode. When on the LAN and running in online (non-cached) mode, there should be no issue connecting to your Exchange server. This is because if RPC over HTTP fails, it should fail over to a standard TCP/IP based RPC connection. If you can’t get your clients to connect while on the LAN, you’ve got larger issues to deal with.

Outlook Connections
You can monitor how your Outlook client is connected to your Exchange server by holding the Ctrl key, right-clicking the Outlook icon in your task bar and selecting “connection status”. You will not see the “connection status” choice unless you hold down the Ctrl key.

Figure 2. Remember to hit the Ctrl key to view the Outlook client connection status menu.

Figure 3. Are you connected vis RCP over HTTPS or TCP/IP? (Click image to view larger version.)

You will get a screen similar to Figure 3 that will show how you are connected in the “Conn” column. You'll either see HTTP or TCP/IP as the connection type. If you see TCP/IP, then you are not using RPC over HTTP or HTTPS.

Try This at Home (only)
You can also begin by testing by not requiring HTTPS for all connections. This will require you to use NTLM authentication for your testing. As mentioned earlier, the recommended way to do this is by using Basic Authentication and over SSL.

To use RPC over HTTP, you will need to make sure that the RPC virtual directory in IIS is not set to require secure connections and that your Outlook clients are set to use NTLM for authentication without SSL. Again, try this on your LAN first. If it works, then move to the security with SSL. If you are able to connect over HTTP and not HTTPS then there may be a problem with your certificate. It is not recommended that you try HTTP from outside of your network for security reasons and the fact that NTLM authentication may not necessarily work with your firewall.

If you are still having trouble getting RPC over HTTPS to work after making sure that your environment and all components meet the requirements, there may be other things to check regarding your overall setup. The excellent KB 827330, “How to troubleshoot client RPC over HTTP connection issues in Office Outlook 2003”, will walk you through many additional troubleshooting steps. I strongly recommend going back and double-checking your setup. Make sure you eliminate problems caused by pre-2003 products as much as possible. Finally, make sure you test on your LAN first without cached mode and monitor how your Outlook clients are connecting to your Exchange server.