jenkinsjenkins-pipelinejenkins-job-builder

I want to know python build in parallel (Jenkins)


Now I'm building using execute shell in Jenkins.

(currently) The code below is built in order. I want to implement this in parallel.

now code status


Solution

  • You have several options to run things in parallel within a Jenkins pipeline.
    The first option is to use the static Parallel Directive Stages which allow you to easily define parallel stages inside your declarative pipeline, something like:

    pipeline {
        agent any
        stages {
            stage('Non-Parallel Stage') {
                steps {
                    echo 'This stage will be executed first.'
                }
            }
            stage('Parallel Stages') {
                parallel {
                    stage('Test 1') {
                        steps {
                            sh "python3 $WORKSPACE/folder/test1.py"
                        }
                    }
                    stage('Test 2') {
                        steps {
                            sh "python3 $WORKSPACE/folder/test2.py"
                        }
                    }
                    .....
                }
            }
        }
    }
    

    A second and more dynamic option is to use the built in parallel keyword which takes a map from branch names to closures:

    parallel firstBranch: {
        // do something
    }, secondBranch: {
        // do something else
    },
    failFast: true|false
    

    and use it to dynamically create your parallel execution steps, something like:

    tests = ['test1','test2','test3', 'test4']
    parallel tests.collectEntries{ test ->
        ["Running test ${test}" : {
            sh "python3 $WORKSPACE/folder/${test}.py"
        }]
    }
    

    This code can reside anywhere in a scripted pipeline, and in a script directive in a declarative pipeline.