Trouble with Cached Exchange Mode in Outlook

Yes, I know we're a little behind. We have not yet migrated to Office 2007 or Exchange Server 2007. Even though we are a small business with only about 100 seats that's still a big financial commitment. Maybe we'll take the plunge next year. But for today, we are still running a very reliable Exchange Server 2003 Enterprise Edition with SP2. Other than needing to occasionally add storage space, it has been working just fine in our organization for the last three years. Management likes that and so do I.

One thing that management doesn't like is Cached Exchange Mode in Outlook 2003. I don't know why it's so much trouble for them. It works fine for me - always has. We may have a scenario that taxes the capabilities of Cached Exchange Mode to the max. In case you don't know, Cached Exchange Mode is simply Microsoft speak for Offline Folders - a local cache of what's in your mailbox on the Exchange Server. We only use it for employees in remote offices or home offices of execs.

Here is the scenario: Executive A is a high-volume, high-density email user in a far city. He easily sends and receives several hundred emails each day, most of them with large attachments of photos or PDFs with embedded photos. We're talking 5 to 10MB of attachments on many of his daily emails. I have never figured out why it has become so acceptable to send such large attachments. It just kind of evolved over the past few years.

Our industry happens to be aircraft sales but the same scenario could exist in Real Estate, automobiles, yachts or any business that needs to send lots of photos back and forth. The executive in question also uses multiple computers - one in the office and one in the home office, both on the East Coast of the U.S. Both his computers are configured to get his email from our Exchange Server on the West Coast of the U.S. using Outlook 2003 and Cached Exchange Mode.

The executive will work all day on the office computer, log off and then work all evening on the home office computer. The complaint is that it will sometimes take hours for synchronization of the offline folders to take place when first firing up one or the other to check his email. He reports that some emails are delayed by many hours while the cache is playing catch up. His mailbox size is over 7GB with over 32,000 individual email messages in multiple folders.

The far city does not have an Exchange Server. We only have the one on the West Coast. All email flows here and then out to the remote office. The remote office is connected via a VPN - a full T1 line here and a 3.1Mbs / 768Kbs DSL line there. The connection speed on the remote home office is a modest 1.5Mbs / 384Kbs DSL with no VPN. Most of the trouble seems to be when connected at the remote home office. Outlook is configured to get email via RPC over HTTP.

I know this reads like an MCSE exam question. Besides cutting his mailbox size down to a more manageable size, what would you recommend?

Update: I posted this same entry on my Tech Republic blog and received many good suggestions and recommendations there. That's why I post most of my stuff on Tech Republic these days. It has a much larger readership of tech guys like me.

Use Small Business Server to full advantage

In the last few weeks I've received several calls asking for assistance with Small Business Server. In each case I have been amazed to discover that the businesses were running SBS either without Exchange Server enabled or with it crippled in some way. Microsoft has done a good job of selling SBS and it is an awesome deal, but apparently the VARs who have been selling and installing it have not done as good a job. Why? Do they not know how to use or configure all the features or is it because they can't convince the SMB to put them in place?

I'm referring specifically to the idea of hosting your own email server. It's not hard to do and I can't figure out why anyone wouldn't want to take advantage of it. The benefits are many. You have total control over your email. You never have to wonder if something got blocked in the spam blocker of your ISP. You can use Outlook Web Access to reach your email from anywhere. You can take advantage of all the benefits of a common shared or Global Address List (GAL). You can use ActiveSync to push your email out to mobile devices.

There are three basic requirements to hosting your email server safely and effectively. First get a great spam filter for Exchange Server. I always recommend Commtouch but you can also go with GFI Mail Security. I have also installed and used Freedom9 Freeguard at some clients. Commtouch is an outside service, GFI Mail Security runs on the Exchange Server and the Freeguard firewall does spam filtering or marking - you can either drop it completly or send it to the mailboxes (my preference) marked as spam and run rules to send it to a spam folder.

The second requirement is to have your ISP change your MX record so all email destined for your domain is sent directly to your Exchange Server. Now do you see why you MUST have a good spam filter in place first? Did you know that 95% of unfiltered email is spam? That can be quite a shock if you have been relying on your ISP to filter your spam for you. My home ISP uses Barracuda and it still struggles even though I've been training it for years. It will send things though that I know I've told it not to and stop things that I have previously cleared.

The third requirement in hosting your own email server is to set up a reverse DNS record with your ISP. This is especially important if you plan to send a lot of emails out through your Exchange Server like a weekly email newsletter to a large mailing list. Without the reverse DNS lookup configured properly with the word mail in their somewhere you will soon end up on the RBL (Real-time Black List) of the spam databases like Spamhaus, Spamcop and Spamcannibal. There are at least 100 spam databases out there. You do NOT want to get listed on any one of them because it a pain to get removed. You can check if you're listed on DNS Stuff.

What do you think? Has my experience been unique or do most SBS users host their own SMTP email on their Exchange Server?

Can you produce emails under legal order?

Awhile back we had a little 'situation' in our organization where we needed to be able to produce copies of emails sent or received by several of our employees. I thought I had it handled and that it would be no problem. I do two backups of our Exchange server each night - one of the entire Information Store (the database) and one of the individual mailboxes (aka brick level).

I have a twenty-day tape rotation and pull a tape once a month so I figured the chances of being able to reproduce the emails would be fairly high. Just pull a tape from the month-end after the time period in question, restore it to a recovery database and viola - there are the emails. The only problem is that the emails weren't there.

What happened? I know they were sent because I could see the headers on my Exchange Server tracking log which I had turned on long ago. I could even see log entries on my SMTP gateway log in Symantec AV for SMTP gateways. I had also turned that log on long ago. I was scratching my head for days all the while under the gun from the boss and the attorneys.

Here's what happened. The employee in question was a high-level executive who had done some social engineering with the IT Manager - me. I got took by a trusted employee because she sweet-talked me into revealing how emails could be permanently deleted in Exchange using a little known feature in OWA - the MS Outlook web client.

As soon as an email was sent or received by the employee that they didn't want tracked, they would delete it and then empty their deleted items folder. Then they would go into the OWA client into the options section and click on the 'View Items' in the 'Recover Deleted Items' section. From there you select the items and then click on 'Permanently Delete'.

You see, normally I have a 30-day window when any employee can recover their own deleted items or I can do it for them. This feature of Exchange is not turned on by default but I have found it very useful. I can't tell you how many times an employee has asked me to help them recover a deleted email before I turned this feature on so they could do it themselves.

If you do the permanent delete right away or at least before the end of the day when I do the nightly backup the items will not be saved. The trick is to catch it before the nightly backup. Otherwise I could still recover them from tape. I would have never revealed that little trick to just any employee but why should I question what a long-term trusted executive asked of me?

Well, that will never happen again. I have now put into place a new archive mailbox and turned on a feature in the Information Store that copies every single piece of email - in or out of the company or even intra-company - to this mailbox. Yes, it grows extraordinarily fast. I have to archive it off to a PST file and purge it at the end of every month or it would be unmanageable.

So now I can produce on demand any email from any employee and any time period even if it was deleted immediately. Yes, it even copies the porn, the jokes, the videos, the personal emails, everything except the spam. 99% of our spam is stopped by Commtouch before it gets to our Information Store. That's a fairly bulletproof backup solution if I say so myself.

The end-user is always right

I got a call today from a user complaining that she couldn't see one of our company contacts in her email address list when composing a new email in Outlook. At first she thought the person in charge of keeping the contact lists up to date had failed in her duties. I was able to see the contact OK and wondered why she couldn't.

It turns out that each Outlook client must be configured at the user workstation to include public contacts as Outlook address lists. Apparently she hadn't done this. Just right click on the public contact folder, select properties, Outlook address book tab, then check the box for 'Show this folder as an email address book'. Do the same for subfolders.

When I advised her of the procedure to do this, she asked, "Did I do something wrong?" I replied, "Of course not. I should have set this up for you in advance." Sigh. That's what computer guys are for, right? I suppose I should put stuff like this on our intranet. It keeps coming up over and over again. But then nobody reads the intranet even when I tell them the answers are there.

Hotfixes - what the publisher calls an 'oopsie'

We got to the bottom of the Symantec Backup Exec 11d problem with random Exchange Server mailboxes not getting backed up properly. It looks like they are being backed up - they just don't show up as a selectable mailbox when you try to restore them. The solution is to make sure you have installed all the latest hotfixes, which requires a reboot of the media server. Oh, and make sure you push out the remote agent to the Exchange Server. This is critical to making it work. Here are the reference documents:

http://seer.entsupport.symantec.com/docs/286823.htm and
http://seer.entsupport.symantec.com/docs/289852.htm

I know, this is boring stuff, but such is the life of an IT Manager. It can get real exciting when an executive calls and wants his deleted mail items restored and you discover that you are unable to do so because you haven't kept your backup software patched.

Backup Exec 11d still not perfect

One of the most basic needs of an enterprise is to ensure good backups. I have been using Symantec / Veritas Backup Exec for years. The product has gone through some growth pains lately. We are up to version 11d now.

We rely heavily on our backups. They are critical to the security of the business and the peace of mind of the IT Manager. I don't have to do a restore very often, but when I do, I sure expect the data to be there in a retrievable format.

One annoying little bug discovered this past week is that some mailboxes do not show up in the restore library selection. It seems that Backup Exec will arbitrarily choose which mailboxes in the database it will include for backup.

One of the first orders of business this week is to figure out the cause of the exclusions - there must be a logical reason, right? The product is fully patched with all the latest hotfixes and otherwise seems to work just fine. Symantec touts their product as being the best for protecting Exchange Server. Right now I'm a little bit disappointed in that claim.