azure-devopsazure-pipelines

Is it possible to re-use build environment dependencies that have been setup, between pipeline runs


I'm new in the Azure DevOps world, normally using Teamcity on our own servers.

I have now have created an Azure pipeline that resolve dependencies before building the application. However, the establishing of the environment takes a long time every time (approx. 10min) and is the environment is the same each time. Is there any way to store the result from the step or cache it ?, so it isn't necessary to create the environment each time.

Just a little context, I'm using uno-check tool to get all needed dependencies setup, so it isn't nuget package, but microsoft workload packages.


Solution

  • After reviewing the uno-check tool, it looks like your concern is about software tooling and frameworks that must be present on the build agent before compilation.

    If you're using the Microsoft-hosted build agents, it's important to recognize that these are temporary agents that exist for the duration of your pipeline run. This ensures that your builds are deterministic and without side effects. These pipeline agents come preloaded with many SDKs and tools, some of which are installed with portability in mind: unpackaged and ready to be used, but missing a configuration setting like a PATH variable which would allow the tool to be resolved at runtime. There are some Azure DevOps pipeline tasks like UseDotNet, JavaToolInstaller or NPM that simply set the PATH to enable the tooling if the version you're requesting is one of the default versions installed on the agent (otherwise they'll install it).

    If the manifest of your uno-check installs the dependencies in a folder that you can control, there is a Cache@2 task that can save and restore that folder between builds. This is typically useful for npm builds where there are lots of third-party components that need to be resolved from external repositories. Updating and restoring the cache is often faster than resolving from external sources, but there is still a sunk time cost for downloading the files to the agent.

    If you have many customizations for the build agent, have network security constraints or want more powerful build agents than the cloud-provided machines, consider using a self-hosted build agent. These agents cost less than the Microsoft-hosted agents, but you'll need to maintain the machine. Microsoft has recently announced Managed DevOps Pools, where you provide the customized disk image but Microsoft hosts the agents on your behalf.

    Another alternative is to use a containerized job. Use a DOCKERFILE to produce an image that contains your required tools, then configure the azure pipeline job to use that container. Note that similar to the Cache@v2 task, the build agent will need to download the image from a container registry each build, but performance is likely to be faster than installing everything. There's a good walk-through here:

    jobs:
    - job: build
      container:
        image: image-name:tag
        endpoint: name_of_container_registry_service_connection
      steps:
      - script: ... # use your developer scripts to compile