jenkins multibranch-pipeline jenkins-generic-webhook-trigger

Jenkins: Generic Webhook Trigger Plugin and Multibranch Scan Webhook Trigger - Issues with genericVariables

Enquiry:

Does anyone else have any ideas? I have tried around a bit in the last few days - unfortunately without any results. The Jenkins log does not give any results either.

Does anyone else have an approach I could follow?

Addendum part 3:

I have pasted the request body I inserted under Addendum part 2 here on a trial basis: https://jsonpath.com/ There I searched again for $.actor.displayName and got the following output:

[
  "Foo Bar"
]

So far so good. I then adapted my pipeline or the Jenkinsfile as follows:

pipeline {
  agent any

  triggers {
    GenericTrigger(
     genericVariables: [
       [key: 'name', value: '$.actor.displayName']
    ],
    causeString: 'Triggered on $ref',
    token: 'test_token',
    tokenCredentialId: '',
    printContributedVariables: true,
    printPostContent: true,
    silentResponse: false,
    regexpFilterText: '$ref',
    regexpFilterExpression: 'refs/heads/' + BRANCH_NAME
    )
  }

  stages {
    stage('Preparing') {
      steps {
        sh '''
        echo Name: ${name}
        '''

When the pipeline is triggered, I get the following output again:

Branch indexing
[some other stuff...]
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Preparing)
[Pipeline] sh
+ echo Name:
Name:
[Pipeline] sh
+ echo 'removing old container: rails'
removing old container: rails

I then tried to output the entire payload by rewriting the pipeline as follows:

pipeline {
  agent any

  triggers {
    GenericTrigger(
     genericVariables: [
       [key: 'name', value: '$']
    ],
    causeString: 'Triggered on $ref',
    token: 'test_token',
    tokenCredentialId: '',
    printContributedVariables: true,
    printPostContent: true,
    silentResponse: false,
    regexpFilterText: '$ref',
    regexpFilterExpression: 'refs/heads/' + BRANCH_NAME
    )
  }

  stages {
    stage('Preparing') {
      steps {
        sh '''
        echo Name: ${name}
        '''

Here too, the result is the same as in the first variant described. Am I still missing something? Shouldn't I get the entire payload with variant 2?

Last but not least, here's a screenshot of my build configuration inside Jenkins. I may have made a mistake there?

Addendum part 2: Performed as POST Request:

Request details:

Event type:repo:refs_changed
URL endppoint:https://jenkins.foo.bar/multibranch-webhook-trigger/invoke?token=test_token

Request header:

X-Request-Id: 686cd082-2bdd-4abb-857a-e46e2d466dfa
Content-Type: application/json; charset=utf-8
X-Event-Key: repo:refs_changed

Request body:

{
    "eventKey":"repo:refs_changed",
    "date":"2023-04-06T14:35:57+0200",
    "actor":{
        "name":"foo@bar.de",
        "emailAddress":"foo@bar.de",
        "id":3,
        "displayName":"Foo Bar",
        "active":true,
        "slug":"foo_bar.de",
        "type":"NORMAL",
        "links":{
            "self":[
                {
                    "href":"https://bb.foo.de/users/foo_bar.de"
                }
            ]
        }
    },
    "repository":{
        "slug":"foo.bar",
        "id":194,
        "name":"foo.bar",
        "description":"Foo bar",
        "hierarchyId":"ec2a64701705b2c36097",
        "scmId":"git",
        "state":"AVAILABLE",
        "statusMessage":"Available",
        "forkable":true,
        "project":{
            "key":"Foo",
            "id":124,
            "name":"Bar",
            "description":"Foo bar",
            "public":false,
            "type":"NORMAL",
            "links":{
                "self":[
                    {
                        "href":"https://bb.foo.bar/projects/foobar"
                    }
                ]
            }
        },
        "public":false,
        "links":{
            "clone":[
                {
                    "href":"https://bb.foobar/scm/foo/bar.git",
                    "name":"http"
                },
                {
                    "href":"ssh://git@bb.foo.bar:7999/foo/bar.git",
                    "name":"ssh"
                }
            ],
            "self":[
                {
                    "href":"https://bb.foo/projects/bar/browse"
                }
            ]
        }
    },
    "changes":[
        {
            "ref":{
                "id":"refs/heads/feature/foo-bar",
                "displayId":"feature/foo-bar",
                "type":"BRANCH"
            },
            "refId":"refs/heads/feature/foo-bar",
            "fromHash":"905944ba5c39999c64e66378613caa1578735780",
            "toHash":"5d7da403f10c07f44aa78f62d4fa8d9d8e5a125f",
            "type":"UPDATE"
        }
    ]

}

Response details:

HTTP-Status: 200
Duration 64ms

Response header:

Date: Thu, 06 Apr 2023 12:35:58 GMT
Server: nginx
X-Content-Type-Options: nosniff
Content-Type: application/json; charset=UTF-8
Connection: keep-alive
Via: 1.1 localhost (Apache-HttpClient/4.5.5 (cache))
Content-Length: 157

Response body:

{
  "status": "ok",
  "data": {
    "triggerResults": {
      "foo/foo.bar/foo.bar Testing": {
        "triggered": true,
        "id": 1163,
        "url": "queue/item/1163/"
      }
    }
  }
}

Addendum:

Here is some more information after I have tried a bit more. The URL that the triggered webhook addresses is structured as follows: https://URL_TO_JENKINS/multibranch-webhook-trigger/invoke?token=test_token

The beginning of the Jenkinsfile looks like this after some adjustments:

pipeline {
  agent any

  triggers {
    GenericTrigger(
     genericVariables: [
       [key: 'name', value: '$actor.name'],
       [key: 'reviewers', value: '$.pullRequest.reviewers'],
       [key: 'ref', value: '$.ref']
    ],
    causeString: 'Triggered on $ref',
    token: 'test_token',
    tokenCredentialId: '',
    printContributedVariables: true,
    printPostContent: true,
    silentResponse: false,
    regexpFilterText: '$ref',
    regexpFilterExpression: 'refs/heads/' + BRANCH_NAME
    )
  }

  stages {
    stage('Preparing') {
      steps {
        sh "echo Ref: $ref"
        sh "echo Name: $name"
        sh "echo Reviewers: $reviewers"
      }
....

In the jenkins console during pipeline execution, I then get the following output:

groovy.lang.MissingPropertyException: No such property: ref for class: groovy.lang.Binding
    at groovy.lang.Binding.getVariable(Binding.java:63)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:285)
    at org.kohsuke.groovy.sandbox.impl.Checker$7.call(Checker.java:375)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:379)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:355)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:355)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:29)
    at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
    at WorkflowScript.run(WorkflowScript:25)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(ModelInterpreter.groovy:137)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.executeSingleStage(ModelInterpreter.groovy:666)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:395)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:393)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.executeSingleStage(ModelInterpreter.groovy:665)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(ModelInterpreter.groovy:288)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.toolsBlock(ModelInterpreter.groovy:544)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.toolsBlock(ModelInterpreter.groovy:543)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(ModelInterpreter.groovy:276)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.withEnvBlock(ModelInterpreter.groovy:443)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.withEnvBlock(ModelInterpreter.groovy:442)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(ModelInterpreter.groovy:275)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.withCredentialsBlock(ModelInterpreter.groovy:481)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.withCredentialsBlock(ModelInterpreter.groovy:480)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(ModelInterpreter.groovy:274)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.inDeclarativeAgent(ModelInterpreter.groovy:586)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.inDeclarativeAgent(ModelInterpreter.groovy:585)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(ModelInterpreter.groovy:272)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.stageInput(ModelInterpreter.groovy:356)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.stageInput(ModelInterpreter.groovy:355)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(ModelInterpreter.groovy:261)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.inWrappers(ModelInterpreter.groovy:618)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.inWrappers(ModelInterpreter.groovy:617)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(ModelInterpreter.groovy:259)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.withEnvBlock(ModelInterpreter.groovy:443)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.withEnvBlock(ModelInterpreter.groovy:442)
    at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(ModelInterpreter.groovy:254)
    at ___cps.transform___(Native Method)
    at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.get(PropertyishBlock.java:73)
    at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
    at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.fixName(PropertyishBlock.java:65)
    at jdk.internal.reflect.GeneratedMethodAccessor188.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:152)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:146)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:146)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:420)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:330)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:294)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
    at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

When I look at the details of the requests and responses that are sent along via the webhook, I also see that the payload is sent along.

If I modify the Jenkinsfile as follows:

pipeline {
  agent any

  triggers {
    GenericTrigger(
     genericVariables: [
       [key: 'name', value: '$actor.name'],
       [key: 'reviewers', value: '$.pullRequest.reviewers'],
       [key: 'ref', value: '$.ref']
    ],
    causeString: 'Triggered on $ref',
    token: 'test_token',
    tokenCredentialId: '',
    printContributedVariables: true,
    printPostContent: true,
    silentResponse: false,
    regexpFilterText: '$ref',
    regexpFilterExpression: 'refs/heads/' + BRANCH_NAME
    )
  }

  stages {
    stage('Preparing') {
      steps {
        sh '''
        echo Reference: ${ref}
        echo Name: ${name}
        echo Reviewers: ${reviewers}
        '''
      }
....

I get the following output:

[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Preparing)
[Pipeline] sh
+ echo Reference:
Reference:
+ echo Name:
Name:
+ echo Reviewers:
Reviewers:
[Pipeline] sh
+ echo 'removing old container: rails'
removing old container: rails

Original question:

I have built a multibranch pipeline in my Jenkins (2.387.1) that I want to trigger via a webhook. A Bitbucket server is connected to Jenkins. The goal is now to trigger a pipeline as soon as all reviewers of a pull request have approved it.

I've installed the Generic Webhook Trigger Plugin. Since this is a multibranch pipeline, I also use the Multibranch Scan Webhook trigger plugin (as also described here: https://plugins.jenkins.io/generic-webhook-trigger/).

With this plugin I managed to trigger the pipeline when the token "test_token" is passed as part of the URL. But since a PR can have several reviewers I want to check within my pipeline (described by a declarative Jenkinsfile) if all reviewers have approved the PR. Therefore, I would like to use the payload of the data sent via the webhook to check whether all reviewers have the PR approved.

Now finally to my problem: Unfortunately, the variable assignment within my Jenkinsfile does not work properly or the payload is empty (for whatever reason). When I output the variables name and reviewers, I only get an empty string.

Here is an excerpt from the beginning of my Jenkinsfile:

pipeline {
  agent any
  // To learn how to use environment variables in Jenkinsfiles, see: https://e.printstacktrace.blog/jenkins-pipeline-environment-variables-the-definitive-guide/
  // environment {
  // }

    triggers {
      GenericTrigger(
       genericVariables: [
        [key: 'name', value: '$.actor.displayName'],
        [key: 'reviewers', value: '$.pullRequest.reviewers']
       ]
      )
    }

  stages {
    stage('Preparing') {
      steps {
        sh '''
        echo Name: $name
        echo Reviewers: $reviewers
        '''

        // Removes previously created containers to enable a clean start of the pipeline before creating the new ones in a further step
        sh '''
        echo "removing old container: rails"
        echo | docker container rm -f rails

        echo "removing old container: maria_test_db"
        echo | docker container rm -f maria_test_db

        echo | docker container ls -a
        '''
....

Can anyone tell me if I have made a mistake here?

Thanks in advance for your help

Solution

Tl;dr: The Generic Webhook Trigger Plugin only works for "simple" pipelines as described on the plugin's website, as far as I understand. Multibranch pipelines can be triggered via the Multibranch Scan Webhook Trigger Plugin, but cannot access or process the payload.

Hello everyone, I have managed to solve the problem in the meantime and did not want to miss to share the "solution" with you, should anyone face the same challenge.

After much back and forth, I think that the combination of Multibranch and the Generic Webhook Trigger Plugin for Jenkins does not work. On the website of the Generic Webhook Trigger Plugin you can currently (09.06.2023) find the following sentence:

This plugin can be used in the jobs created by the Pipeline Multibranch Plugin. If you are looking for a way to trigger a scan in the Pipeline Multibranch Plugin you can use the Multibranch Scan Webhook Trigger Plugin.

For me, this sentence was somewhat misleading, as the pipeline can be triggered via the plugin, but there is no way to read the payload or access it within the pipeline.

My solution now was to map my CI/CD process with "simple" pipelines, which are then passed along in the webhook via tokens. In "simple" pipelines, the Generic Webhook Trigger Plugin works perfectly.