fedora-25concourse

concourse resource - task output not available to put


How do I put the output of a task into a resource?

  - name: build-pkg-rpm
    public: true
    plan:
    - aggregate:
      - get: oregano-test-fedora
      - get: git-clone-resource
        trigger: true
        passed: [compile, build-docker-image-fedora]

    - task: create-rpm
      image: oregano-test-fedora
      config:
        platform: linux
        inputs:
        - name: git-clone-resource
        outputs:
        - name: srpm
          path: ../srpm
        run:
          path: .concourse/fedora/buildrpm.sh
          dir: git-clone-resource

    - put: srpm
      resource: copr-resource
      params:
        rpmbuild_dir: "srpm/rpmbuild/SRPMS"
        chroots: ["mageia-6-x86_64", "mageia-couldron-x86_64", "fedora-rawhide-x86_64", "fedora-25-x86_64"]
        enable_net: false
        max_n_bytes: 250000000
        project_id: 825
        regex: ".*oregano-.*\\.src\\.rpm$"

buildrpm.sh

#!/usr/bin/env bash

set -e
set -x

pwd 2>&1
RPMBUILD_DIR="$(pwd)/../srpm/rpmbuild/"
mkdir -p ${RPMBUILD_DIR}/{SOURCES,BUILD,RPMS,SRPMS,SPECS}

# fill all vars of the spec.in
./waf configure rpmspec
cp -v build/rpmspec/oregano.spec ${RPMBUILD_DIR}/SPECS/

# generate the distributable tar
./waf dist
cp -v oregano*.tar.xz ${RPMBUILD_DIR}/SOURCES/

cd ${RPMBUILD_DIR}
rpmbuild \
--define "_topdir %(pwd)" \
--define "_builddir %{_topdir}/BUILD" \
--define "_rpmdir %{_topdir}/RPMS" \
--define "_srcrpmdir %{_topdir}/SRPMS" \
--define "_specdir %{_topdir}/SPECS" \
--define "_sourcedir  %{_topdir}/SOURCES" \
-ba SPECS/oregano.spec && echo "RPM was built"
pwd 2>&1

According to

https://github.com/starkandwayne/concourse-tutorial/tree/master/12_publishing_outputs

this should work, yet the directory in the put step is empty.

The documentation https://concourse-ci.org/put-step.html seems to not cover this topic very much.

I see that the files are writte properly:

Wrote: /tmp/build/be0f50d1/srpm/rpmbuild/SRPMS/oregano-0.84.3-1.fc25.src.rpm Wrote: /tmp/build/be0f50d1/srpm/rpmbuild/RPMS/x86_64/oregano-0.84.3-1.fc25.x86_64.rpm Wrote: /tmp/build/be0f50d1/srpm/rpmbuild/RPMS/x86_64/oregano-debuginfo-0.84.3-1.fc25.x86_64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.6mO3iF
+ umask 022
+ cd /tmp/build/be0f50d1/srpm/rpmbuild/BUILD
+ cd oregano
+ rm -rf /tmp/build/be0f50d1/srpm/rpmbuild/BUILDROOT/oregano-0.84.3-1.fc25.x86_64
+ exit 0
+ echo 'RPM was built' RPM was built RPM was built
+ pwd /tmp/build/be0f50d1/srpm/rpmbuild

But then when it comes to put the task's output, it does not find any files:

base dir: "/tmp/build/put/srpm/rpmbuild/SRPMS"
error: Could not find any matches with that regex
caused by: WalkDir entry is useless
caused by: IO error for operation on /tmp/build/put/srpm/rpmbuild/SRPMS: No such file or directory (os error 2)
caused by: No such file or directory (os error 2)

When I hijack into the containers:

1: build #136, step: create-rpm, type: task
2: build #136, step: srpm, type: put

choose a container: 1

[root@f004c6a0-adee-4735-792f-8e237f645751 be0f50d1]# ls -al
total 0
drwxr-xr-x. 1 root root  44 May 31 11:43 .
drwxr-xr-x. 1 root root  24 May 31 11:43 ..
drwxr-xr-x. 1 root root 598 May 31 11:43 git-clone-resource
drwxr-xr-x. 1 root root  16 May 31 11:43 srpm
[root@f004c6a0-adee-4735-792f-8e237f645751 be0f50d1]# ls -al srpm/rpmbuild/
BUILD/     BUILDROOT/ RPMS/      SOURCES/   SPECS/     SRPMS/     
[root@f004c6a0-adee-4735-792f-8e237f645751 be0f50d1]# ls -al srpm/rpmbuild/
BUILD/     BUILDROOT/ RPMS/      SOURCES/   SPECS/     SRPMS/     
[root@f004c6a0-adee-4735-792f-8e237f645751 be0f50d1]# ls -al srpm/rpmbuild/SRPMS/
total 1232
drwxr-xr-x. 1 root root      58 May 31 11:43 .
drwxr-xr-x. 1 root root      70 May 31 11:43 ..
-rw-r--r--. 1 root root 1260666 May 31 11:43 oregano-0.84.3-1.fc25.src.rpm

choose a container: 2

/tmp/build/put # ls -al
total 0
drwxr-xr-x    1 root     root            82 May 31 11:43 .
drwxr-xr-x    1 root     root             6 May 31 11:43 ..
drwxr-xr-x    1 root     root           414 May 31 11:31 git-clone-resource
drwxr-xr-x    1 root     root           130 May 23 07:25 oregano-test-fedora
drwxr-xr-x    1 42949672 42949672         0 May 31 11:43 srpm
/tmp/build/put # #ls -al srpm/
/tmp/build/put # #ls -al srpm/
/tmp/build/put # #ls -al ../srpm
/tmp/build/put # ls -al ../srpm
ls: ../srpm: No such file or directory
/tmp/build/put # ls -al srpm
total 0
drwxr-xr-x    1 42949672 42949672         0 May 31 11:43 .
drwxr-xr-x    1 root     root            82 May 31 11:43 ..

So why is the file structure nor oregano-*.src.rpm available in the put step?

The full concourse YAML is avail here, though not necessary as far as I can tell https://github.com/drahnr/oregano/blob/master/.concourse.yml


Solution

  • The issue was this:

    - task: create-rpm
      image: oregano-test-fedora
      config:
        platform: linux
        inputs:
        - name: git-clone-resource
        outputs:
        - name: srpm
          path: ../srpm
    

    path: ../srpm placed the outputs outside the volume and as such the content was lost. Note that dir does not do anything regarding the inputs and outputs but just changes the workdir for the execution of the script!

    The default would have sufficed which would have equaled to path: srpm (or path: "")

    This works:

    - task: create-rpm
      image: oregano-test-fedora
      config:
        platform: linux
        inputs:
        - name: git-clone-resource
        outputs:
        - name: srpm