cmakegithub-actionsconan

Installing dependencies with conan on github actions takes too long


I have built a Github workflow that installs Conan to manage a bunch of C++ dependencies along with CMake. It works great and I can test that:

  1. my environment works
  2. my library builds
  3. tests pass

at every commit.

However, it is not very convenient for repeated pushes, as the Conan install dependencies step takes too long. Is there a way to:

  1. Bypass the Conan installation and reuse previous workspace if dependencies are unchanged (that is, most of the time)
  2. If not, what is the "standard" to have an equivalent system configuration in place on my system for tests (Does Conan has a Python virtual environment equivalent, or should I use a Docker, or ... ?)

My workflow is the following:

name: CMake

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

env:
  # Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
  BUILD_TYPE: Release

jobs:
  build:
    # The CMake configure and build commands are platform agnostic and should work
    # equally well on Windows or Mac.
    # You can convert this to a matrix build if you need cross-platform coverage.
    # See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: Get GCC-10
      run:   |
             sudo apt update
             sudo apt install gcc-10 g++-10
      shell: bash

    - name: Install Doxygen
      run: sudo apt-get update && sudo apt-get install -y --no-install-recommends doxygen graphviz

    - name: Conan installation
      id: conan
      uses: turtlebrowser/get-conan@v1.0

    - name: Conan version
      run: echo "${{ steps.conan.outputs.version }}"

    - name: Create default Conan profile
      run: conan profile new default --detect

    - name: Conan configure remotes
      run:  |
            conan remote add bincrafters https://bincrafters.jfrog.io/artifactory/api/conan/public-conan
            conan config set general.revisions_enabled=1

    - name: Conan profile update
      run:  |
            conan profile update settings.compiler=gcc default
            conan profile update settings.compiler.version=10 default
            conan profile update settings.compiler.cppstd=20 default
            conan profile update env.CC=[/usr/bin/gcc-10] default
            conan profile update env.CXX=[/usr/bin/g++-10] default
            conan profile update conf.tools.cmake.cmaketoolchain:generator=Ninja default
            conan profile show default

    - name: Conan install dependencies
      run:  |
            conan install . \
            -s build_type=${{env.BUILD_TYPE}} \
            --install-folder=${{github.workspace}}/build \
            -pr default \
            -b=missing

    - name: CMake configuration
      # Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
      # See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
      run:  |
            cmake -B ${{github.workspace}}/build \
            -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake \
            -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
      shell: bash
      env:
       CC:   gcc-10
       CXX:  g++-10

    - name: Cmake build
      run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}

    - name: Cmake test
      working-directory: ${{github.workspace}}/build
      run: ctest -C ${{env.BUILD_TYPE}}

    - name: Doxygen documentation generation
      working-directory: ${{github.workspace}}/build
      run: make doc

    - name: Documentation moving generated files
      run: mv ${{github.workspace}}/build/doc/html ./doc/api

    - name: Documentation deploy to Github Pages
      uses: peaceiris/actions-gh-pages@v3
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        publish_dir: ./doc/api

Solution

  • You can use the GitHub actions cache to cache the .conan2 folder to speed up your workflow.

    As per the Github Actions Caching Dependencies Documentation, you can add the following to your workflow:

    - name: Cache Conan Dependencies
      id: cache-conan
      uses: actions/cache@v2
      with:
        path: path/to/.conan2
        key: # choose a key here
    # if we did hit the cache, then you do not want to create a new default  
    # profile since you already have one
    - if ${{steps.cache-conan.outputs.cache-hit != 'true'}}
      name: Create default Conan profile
      run: conan profile new default --detect
    

    after the Conan Version Section.

    For the key, normally you would hash your dependencies file using hashFiles() to get a unique key. I do not know if you are using conanfile.txt, conanfile.py or neither. If you are you can do use hashFiles('**/conanfile.txt') or hashFiles('**/conanfile.py') to generate a key.

    If you are not using either you will need to find a way of generating a unique key for your cache that changes when you update your dependencies.