Wrestling with the Windows Installer
A Visual Studio .NET Setup project looks good at first, but there are plenty of traps for the unwary, and some features require external tools. By Tim Anderson.
Visual Studio includes integrated support for creating setup applications so that you can easily deploy your .NET applications. It is disappointing that it doesn't know how to deploy the .NET Framework runtime - a problem fixed in the forthcoming 2005 release - but it still has some nice features. It is in effect a visual editor for Microsoft Installer databases, the .MSI files you will be familiar with if you work with Windows. An installation engine that understands .MSI files is built into Windows, including automatic repair and some great features for administrators rolling an application out across an entire network. The Microsoft Installer is therefore is the officially preferred approach to Windows deployment.
At first glance Visual Studio is a decent MSI editor. I did discover that the dependency detection is not reliable, and for a while I was plagued with duplicate assemblies appearing by magic, but there's a tip to fix this: remove the SearchPath, which is a property of the setup project, and dependency detection will no longer bother you. It is also important to read up on a few critical installer features. At a minimum, you need to understand the meaning of the UpgradeCode, ProductCode and PackageCode, and the implications of Version, RemovePreviousVersions and DetectNewerInstalledVersion. You learn that you never ever change the UpgradeCode, and that Version must be incremented if you want the setup to install over the top of an existing version of your app, and that you should let Visual Studio generate new ProductCode and PackageCode GUIDs when it wants to. You also learn to beware of installing things like database or configuration files with your setup. Here's how it can go wrong. You install somedata.mdb and someconfig.ini with the first cut of your application. Works great. Now here comes the next version. If you have RemovePreviousVersions set, the installer will handily delete the user's database and config files before installing brand new versions. Probably not what you want. You can get round this by setting the Permanent property to true for a particular file, which stops the Installer from removing it, and then setting a Not Installed condition so that it doesn't get overwritten. However, there are bugs in some versions of the Installer that make the Permanent setting ineffective. Bottom line - don't go there. I resorted to a little custom action that copies the files I never want to overwrite seperately from the installer engine.
Ah, custom actions! These guys let you do all kinds of things that would otherwise be impossible. Visual Studio even has a Custom Action editor, which you get to by choosing View - Editor - Custom Actions when a setup project is active. I used this to fire up my utility for copying data files, but I ran into a more difficult issue. I decided it would be nice if users had the option to run the application automatically at the end of the install. Step one was to add a dialog with a checkbox offering the option, and that worked fine. The dialog, that is. I could not get the app itself to run at the right moment, that is, right at the end of the install. The best I could manage was to have it run before the end of the install, which is horrible.
Let me introduce you to Orca. This natty utility is installed with the Windows SDK; it is another MSI editor. Unlike Visual Studio though, Orca gives you the raw unsullied view of the Installer tables. So how many tables does it take to install a few files? A lot, apparently. Another snag is that MSI is a relational database, and many of the keys are GUIDs. Orca gives a table by table view, and working out which GUID in one table applies to which file in another table is a bit of a strain. Still, anything you can do with the Installer, you can do with Orca. After a bit of Googling, and a bit of ferreting through the Installer SDK, I worked out how to create a custom action of type 210 that would fire up my application, and added a DoAction to the ControlEvent table that runs the custom action when the user clicks Close on the last dialog. There are a few other tweaks I made in Orca that I won't go into now. Enough to say that by tweaking my install with Orca I could get it running as I wanted.
Orca - no prizes for UI, but still a powerful MSI editing tool. Can you spot that the Dialog column actually has a trailing underscore?
However, there is a flaw in the Orca plan. Every time you build your setup, your changes are lost. When you get to the point where you are adding or editing 6 or 10 lines in Orca to make your install work properly, you find you have introduced a tedious and error-prone extra step. Clearly I had to find a way to automate the process. You can't automate Orca itself, but you can use the same API that Orca uses. This is an unmanaged API, so I started up VB.NET, dug out MsiQuery.h from the Platform SDK, and got to work adding PInvoke declares for the main MSI functions. Google suggests that while I am not the first to do this, there are not many others that have gone down this route. There's actually an example in MSDN, but it is not very helpful. It shows how to open an MSI database in code, and how to close it. Nothing about doing anything useful in between.
The MSI API doesn't look too bad at first. It understands SQL, so I figured all I needed to do was to fire some SQL queries and statements to make the changes I needed. I know SQL pretty well, but this didn't prove to be much advantage. First, I found I could execute SELECT * FROM sometable, but not much else. I kept getting an invalid syntax error, for queries that looked fine to me. To be more accurate, I got error 1615 which is the standard Windows error code for ERROR_BAD_QUERY_SYNTAX. Well, it turns out that MSI has some special SQL requirements. To have success, you need to surround field and table names with grave accents, as in `ControlEvent`.`Action`. All the names are case-sensitive too. Once I discovered this, things went better, but I was still getting syntax errors on a particular table. As the night wore on, I was studying the table in Orca, comparing the column names with those in my query, not finding any errors and getting more and more frustrated. Then I looked up the table in the SDK. I made the painful discovery that two of the column names have trailing underscores, as in "Dialog_". In Orca, the trailing underscore is pretty much invisible.
Most things now worked, but I was still getting syntax errors on a particular update statement. I narrowed it down to the column I was trying to update, which was VersionMin in the Upgrade table. I could update other columns such as ActionProperty without any issue, but not VersionMin. This time the error was 1627, also known as ERROR_FUNCTION_FAILED. Indeed. In desperation, I replaced one line of SQL with several. Retrieve and store the column values, delete the row, build the SQLto insert a new row, insert it. To my relief, that worked. I still don't know why the other Update kept failing. By the way, even if you open an MSI database with MSIDBOPEN_DIRECT, rather than MSIDBOPEN_TRANSACT, you still usually have to call MsiDatabaseCommit before changes are written to the file. Just thought I'd mention that.
Finally, I noticed a problem that arises if the Installer finds some files in use and prompts for a reboot. This occurs after the normal last dialog. Unless you take steps to prevent it, your app will attempt to fire up at the same time as the Installer is asking for a reboot. So I added a condition that checks the ReplacedInUseFiles property, which indicates whether the installer will ask for a reboot. Didn't work - because that property is not availalble in the UI sequence. Another smart installer feature. So I added a new global property of my own along with a custom action consisting of one line of VB Script. This action checks the Session.Mode(6) property, which also indicates if a reboot is needed. If it is, the VB code sets my new property to "Yes". I then use my global property as a further condition before executing my app.
So now I'm done. I just build the setup project, then run my VB.NET application to modify the tables. What's more, it would be trivial to make further modifications, encouraging me to explore further. But after all this, I guess the obvious question is: why did I bother? Why not just use InstallShield, or any one of several more straightforward and intuitive installation tools, some of them free?
I'm not altogether sure, but here's a few considerations. First, I like the idea of using the proper Windows installer service. Second, I tend to the view that less is better, and I'm happy to be without the extra layers that the likes of Wise and InstallShield impose on Installer packages. Third, having made this effort I'm hoping that everything will now work smoothly; certainly the build process is now very manageable. Fourth, I still enjoy a challenge.
A few questions remain. The chief one is: why does Microsoft make it so difficult? Certain tasks are easy enough, such as editing the registry, creating shortcuts, and of course copying a bunch of files. Other tasks are near-impossible, unless you take steps such as those outlined above. Another challenge is discovering which is which. Where is the document that clearly lays out what you can do with a Visual Studio setup project, and what are its limitations? Here's hoping for better things in Visual Studio 2005.
Copyright Tim Anderson 9th September 2004. All rights reserved.