Subtle MSBuild bug (feature)

I've got a fairly complex MSBuild script that has a subtle problem that has taken me a while to diagnose. The symptoms where basically that the first "fixed" build would often fail. What I mean by this is that after a compile problem was fixed the next build would fail to create an *.msi file because it said there were missing files. Here is an outline of my build process:

1. Get the latest code from Source (Un)Safe.
   


2. Clean (delete all bin and obj directories).
   


3. Compile 65 VB.NET 2005 projects individually in dependency order (NOT using a master solution file).
   
   
4. Copy most of the build output to a common bin directory.
   


5. Run an InstallShield script that packages the files in the common bin directory into an *msi file.
   


6. Copy the resulting *.msi file to a turnover directory.
   


7. Email QA to let them know there is a new build available.

Here is a simplified version of the MSBuild script for step 4:

<Target Name="CompileWithStop">

<MSBuild

Projects="@(ProjectReferences)"

Targets="Build"

StopOnFirstFailure="True" />

</Target>

<ItemGroup>

<CompileOutput Include="..\Source\**\bin\**\*.exe" Exclude="..\Source\**\bin\**\*.vshost.exe" />

</ItemGroup>

<Target Name="Copy" >

<Copy SourceFiles="@(CompileOutput)"

DestinationFolder="$(OutputDirectory)"></Copy>

</Target>

And here is the bug: It turns out that all ItemGroups in an MSBuild script are processed BEFORE all Targets! So this means that my CompileOutput item actually is based on the files that were generated by the PREVIOUS build! The insidiously subtle part of this is that builds usually generate the same output, so the script "works" except for the first time new file is added, or the first time after a broken build that didn't generate all of it's output files.

Fortunately there is a solution. The CreateItem task can be used to create an Item within a Target. The MSBuild snippet below fixed my problem:

<Target Name="Copy" >

<CreateItem Include="..\Source\**\bin\**\*.exe" Exclude="..\Source\**\bin\**\*.vshost.exe">

<Output TaskParameter="Include" ItemName="CompileOutput" />

</CreateItem>

<Copy SourceFiles="@(CompileOutput)"

DestinationFolder="$(OutputDirectory)"></Copy>

</Target>

This "feature" is apparently "by design", but it is definitely counter intuitive. I had incorrectly assumed that an ItemGroup was processed when (and only IF) it was referenced...

Here is the reference that solved it for me: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=390321&SiteID=1