jenkinsjenkins-pipelinehigh-availabilityjenkins-agent

How to choose between two agents for high-availability in Jenkins pipeline


I have several agents configured in Jenkins.

For one of my pipeline jobs execution, I wish to choose between two of my agents i.e. MYHOST11-ANSIBLE-AGENT and MYHOST22-ANSIBLE-AGENT whichever is available. Thus, if MYHOST11-ANSIBLE-AGENT is unavailable my Jenkins pipeline job should switch to using MYHOST22-ANSIBLE-AGENT

Can you please suggest what changes do I need in my below pipeline code ?

pipeline {
    agent { 
        node {
            label 'MYHOST11-ANSIBLE-AGENT' 
        }
    }
    stages {
        stage('Precheck') {
            steps {
                sh "echo Im from Jenkins>/tmp/jenkinsmoht.txt"

Note: I want my pipeline to choose only between the two agents I mentioned as only they have ansible with my pipeline invokes. Other agents don't have ansible thus my pipeline would fail.


Solution

  • I think you have your nomenclature strategy a little skewed...

    Let's say you have two nodes, named "HOST11" and "HOST22". Those two have ansible installed. Other nodes (eg "HOST33") do not. Those are the names of the Nodes, reflective of the Hosts the agents run on.

    You want to configure your Nodes (/computer/<NodeName>/configure) with "labels" according to their characteristics, in this case "ansible", thus creating a "pool of nodes" of similar configuration.

    Node Config

    You then use the label of the characteristic ("ansible") to assign the job to the pool of servers with that corresponding label (and characteristic). By assigning labels to nodes, you can specify the resources you want to use for specific jobs, and set up graceful queuing for your jobs.

    eg:

    agent { 
        node {
            label 'ansible' 
        }
    

    Jenkins will then pick the first available node with the matching label and run there, unless that node is not available. It will then try the next. If none are available, the job will remain queued.

    If you choose to use a Host Name (which in truth is also just a "label", then you can only run on that one node).

    Another distinction: "Available Node" in Jenkins implies on-line. If all executors are busy, it is still "available". Jenkins jobs are "sticky", it will wait until a node it has previously run on has an available executor. That can also result in the first node being overloaded. If that's a problem, look to install the "Least Load" plugin which will act as a load balancer using various criteria.

    See this post to further constrain jobs using multiple labels.

    ps: If your nodes are similar but not identical, you may be use the "Node Properties" or the "Slave Setup" plugin to make them transparently compatible to your job (eg: set VAR to different values/paths).