A Windows technology I love to hate is the Windows Installer, the setup engine built into Windows that is hard to avoid if you want to comply with Microsoft logo requirements and system management tools.

I have a little application which uses this, which uses a custom action written in Delphi. Originally this was an executable with some command-line arguments, which worked fine except that occasionally the custom action needs to show a dialog. Sometimes (not always) this would show up behind the main setup window, causing users to think that setup had hung.

Incidentally, I saw this exact problem when installing Delphi itself on Windows Vista.

It is all to do with a long-standing and complex Windows issue concerning whether applications can force a window to be on top of other windows. In a nutshell, a well-behaved application should not normally do this, though it can make itself flash in the taskbar. Not a great user experience. However, you can ensure that a window is on top within a specific application (in this case the setup), provided you know the handle of that application’s main window. Unfortunately there is no obvious way to get this value, other than via an API called FindWindow which might occasionally find the wrong window, for example if the user managed to open two instances of the setup.

The correct solution for this is not to use an EXE as a custom actions, but rather to use a DLL. This runs in-process with the setup, which enables it to call MSI (Microsoft Installer) functions like MsiProcessMessage, enabling it to show dialogs safely. You can also do useful things like writing entries to the installer log. (Thanks to Mike on the Microsoft.public.windows.msi newsgroup for this tip).

Therefore I converted the custom action to a DLL. Not too difficult; but I discovered that the Windows Installer is not especially flexible about calling custom actions in DLLs. The only argument it can (and must) give is a handle to itself. That’s unlikely to be enough. So how do you pass data to your custom action?

If you use an MSI editor such as that in Visual Studio, you will see a property called CustomActionData which you can set when calling a custom action. All this does is to set a property within the installer. Your custom action can then call MsiGetProperty to retrieve the value. It is a single string; if you want to pass several values, you need to use some sort of delimiter and parse it within your custom action code.

Although many Windows API functions have Delphi wrappers built into Delphi’s runtime library, the Installer functions are not among them. I hate reinventing the wheel, so I searched for a Delphi wrapper for msi.dll. It’s not easy to find, suggesting that few developers have gone down this route, though it is part of some versions of the Project JEDI JCL (JEDI component library). In the end it was easier to find the header files in the SDK and do my own wrapper for the few functions I needed.

Delphi is a great tool, but at times like this you realise that there is a price to pay for not falling in with the crowd and using Visual C++. All the low-level Windows API documentation assumes that you are at least using C++.

The good news: it all works fine. Dialogs appear reliably above the setup window, and access to the MSI API may prove useful for other things as well.

That said, it all goes to demonstrate why developers sometimes take ages to fix seemingly simple problems.