c++windowsazure-devopsmsbuildincremental-build

Are incremental builds possible with copied files not built on the machine?


I'm having trouble setting up incremental builds in Azure DevOps. There are too many variables with workspace cleaning to ensure that I don't have to do a full build every time.

I had a thought that I could just always copy the built files to a location outside of the agents' purview, and then copy those files into my release directory before each build.

Would that allow for an incremental build?


Solution

  • You probably can 'fool' the incremental logic but you would be working against the tooling.

    For an actual incremental build you need to build in the same place.

    In the context of Azure DevOps, that means building the same job of the same pipeline on the same agent. You can't let the build move around between agents or even between work folders of the same agent. (It also means that your agent and the state of the agent work folder must be persistent across the builds.)

    You can make the job, stage, or pipeline 'sticky' to one dedicated agent by using demands and capabilities.

    Decide what will be on your dedicated agent. Will it be the entire pipeline or just a stage of the pipeline or just a job of a stage?

    For the dedicated agent, create a capability that represents the build. Using the name of the pipeline (or pipeline+stage or pipeline+stage+job depending) for the name of the capability is handy and self-documenting. You can create the capability in Azure DevOps as a 'user capability' of the agent.

    Change your pipeline to add a demand on the custom capability. The demand can test if the custom capability exists. In a YAML pipeline the demands are configured in the pool definition.

    This is an easier and less brittle approach then trying to outsmart the incremental logic.

    With this approach, all builds will be done in series on the one agent. If the build takes a long time (which may be the motivation for building incrementally) and the build is tied to one agent, the 'throughput' of builds will be limited. If a build's duration is 1 hour, there will be a maximum of 8 builds in an 8 hour work day.

    Tying specific builds to specific agents is not the intent in Azure DevOps. For a monolithic legacy codebase where there is no notion of semantic versioning and immutable interfaces, you may have little choice. But a better way is to use package management. Instead of one big build, have multiple smaller builds that produce packages that are used by other builds. The challenge is that packages will not work well without some attention and discipline around versioning and keeping published interfaces and contracts unchanged.