cachinggitlab-cigitlab-ci-runnerartifacts

Multiple paths with different expiry time in gitlab-ci runners

I'm having issues with parsing a build directory between stages using Gitlab-CI.

Gitlab-CI wipes the created build artifacts between stages which seems weird. I'm trying to store the build artifacts between the build and the test stage, however the build stage also has the build artifact which I want to keep and also the build artifacts which are required to run the next stage.

Is it possible to have multiple expiry times with different paths using the artifacts option?

I have tried the following, which only keeps the second definition of paths (the build/test* paths), and not the first paths (.dmg) declared.

  artifacts:
    paths:
    - build/*.dmg
    expire_in: 1 week
    paths:
    - build/test1
    - build/test2
    - build/test3
    expire_in: 15 mins

I have tried using the caches however can't seem to get that working... Any suggestions would be great appreciated!

Solution

  • According to the docs it doesn't seem possible but I needed to do something similar so as a workaround I did the following.

    In the stage of the build that generates all the artifacts I set an expiry of 15 minutes for all the artifacts including the one that should have a different expiration. In your case build/*.dmg. So my artifacts definition for the build section would be like this:

     artifacts:
    paths:
    - build/*.dmg
    - build/test1
    - build/test2
    - build/test3
    expire_in: 15 mins

    After that I defined two jobs in next stage of the pipeline. If you do this the two jobs run concurrently. One of the jobs does what you originally intended for the files: build/test1, build/test2 and build/test3. The other job in this stage of the pipeline should have practically nothing in the script section, maybe something like echo "dummy job". But because the dummy job receives the build artifacts from the previous job we can use it to 'change' the expiry from 15 min to a week by simply creating a second artifact with the same file like so:

    artifacts:
    paths:
    - build/*.dmg
    expire_in: 1 week

    It is a poor workaround because it duplicates the target artifact, since we'll have one with an expiration of 15min and another with an expiration of 1week.