When I ran the service on a clean XP SP3 system, I got
The system cannot execute the specified program.
Googling this phrase, it implied we had a missing dependency (DLL), so I ran Dependency Walker but found no missing dependencies.
However, Dependency Walker did find an error, giving me another set of phrases to Google:
Error: The Side-by-Side configuration information for "foo.exe" contains errors. This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem (14001)."Side-by-Side configuration information" is the clue that this is actually a problem with the application manifest. Now we're getting somewhere...
Warning: At least one delay-load dependency module was not found.
Warning: At least one module has an unresolved import due to a missing export function in a delay-load dependent module.
As it turns out, the Visual Studio 2005 project was configured to have the linker generate the application manifest, but for some reason (I haven't root caused it yet), the manifest being generated is wrong: it specifies a dependency on the VC8 runtime, but the service is actually statically linked to the runtime to minimize dependencies.
So, the reason we're seeing this vague error when we run the service is because the application manifest is horked. When I rebuilt the service without the application manifest, the problem "went away."
The next step is to take a look at the manifest and see what it ways.
You'd think looking at an application manifest would be simple: it isn't.
Fortunately, the indispensable Kenny Kerr has written a tool which will display the application manifest: Manifest View.