I had a call last night from a small business whose email no longer worked. They had applied updates to the server but Exchange had failed to restart.
Looking at the services it was easy to see why. All the Exchange services and certain others including the IIS web server were set to disabled:
The likely culprit was Update Rollup 5 for Exchange Server 2007 Service Pack 3 (KB 2602324) – or rather, the mechanism which applies the patch, since this seems to be an issue that others have run into as far back as 2008 with other Exchange patches, though it is rare:
I installed the Update Rollup 4 and did a reboot of my Exchange Server 2007. But since then, all my services are disabled. Is this a known issue?
My guess is that the patch disables the services in order to update the binaries and then, for some unknown reason not fixed by Microsoft over these last four years, fails to re-enable them.
It seems that no harm was done other than that the services were disabled, but how can you know which services are meant to be running, which should be set to manual, and which should stay disabled?
I contemplated doing a quick test install of SBS 2008 on a VM just to see how it is set out of the box, but fortunately found this post by Susan Bradley which shows default SBS 2008 running services.
There were a few other things wrong. SharePoint Services was raising event 5586:
Unknown SQL Exception 33002 occured. Additional error information from SQL Server is included below. Access to table dbo.Versions is blocked because the signature is not valid.
and there was the related event 33002 from the internal SQL Server used by SharePoint. The cause of this was SharePoint Services 3.0 Service Pack 3. When you apply a major update to SharePoint Services, you have to re-run the SharePoint Products and Technologies Configuration Wizard. This is by design, though it seems odd to me that you apply an update and it silently breaks the product it is updating until you run a further manual process. Of course the error itself does not give you much clue about what is really wrong.
The third major issue was a JRNL_WRAP_ERROR from the NTFrs File Replication Service. You have to be careful with this one, since the advised fix in the event log presumes the presence of a good replica elsewhere, which in the case of SBS is unlikely. See this article for details. With SBS which it is the sole domain controller you should set the BurFlags registry key to D4. Further comment on ServerFault here.
The incident reminds me of how prickly SBS can be. It is great value for what it does, but has all the complexity of Microsoft’s server stack plus the further disadvantage of being crammed onto one machine. I prefer a pseudo multi-server approach, even for small businesses, with at least two physical servers and separate VMs for Exchange, SharePoint, domain controller, backup DC on the other physical machine, and so on. Of course this has complexity of its own.
I would guess that when upgrade time comes around, companies like this will be looking carefully at Office 365. Or Google Apps; but the advantage of Office 365 is that you can make the transition from SBS with relatively little impact on users: just migrate the Active Directory, Exchange and SharePoint. You lose flexibility and some local performance, but hand over the maintenance issues to Microsoft.
You hit the nail on the head. We are terrified of our SBS 2008 (though less terrified of it than we were of SBS 2003). The migration from 2003 to 2008 is not something that I would like to repeat again. At replacement time, we’ll be moving e-mail to Office 365 or Rackspace Exchange, and then just setting up a vanilla domain controller on a local machine. (We don’t use SharePoint, which broke itself some time ago anyway.)
It’s gotten a bit better with 2008, but it’s still somewhat true: I remember that it was possible to take a vanilla SBS 2003 and watch it destroy itself with Windows Updates.
The company has been running some sort of SBS for nearly a decade, and the best reason that I can come up with for this is that it made sense at the time: e-mail is a vital company asset, and at the time the best security option for that meant having it on site. But with reliable Exchange offerings in the cloud, with the ubiquity of offsite backup solutions, and with the complexities of managing an e-mail server in general (blacklists, SPF, anti-spam updates, complaint feedback loops, oh my!), I see less and less reason for a small business to task itself with the infrastructure cost of maintaining local Exchange server.