powershellexpectrundeck

Unable to get expect scripts to work in Rundeck


I have a fresh install of Rundeck on one of our management VMs. I used Rundeck at my previous employment so I have some history using the tool.

Background: We have multiple GitHub repositories containing code and we push updates to scripts that we want Rundeck to run. I have previously been able to write scripts that call other scripts without issues. I have been able to use expects scripts together with powershell scripts before to push new passwords or files over SSH via Rundeck without issues.

Rundeck job config: enter image description here I have tries multiple ways to try to execute the script without success.

Powershell script:

#!/opt/microsoft/powershell/7/pwsh

. /var/lib/rundeck/github/redacted/modules/netbox.ps1
. /var/lib/rundeck/github/redacted/modules/keys.ps1
. /var/lib/rundeck/github/redacted/modules/tools.ps1


function run_rkhunter {
    param (
        [Parameter(Mandatory = $true)]$ssh_ip,
        [switch]$view_log
    )

    $prompt = . /var/lib/rundeck/github/redacted/scripts/rkhunter.exp "$ssh_ip"

    if ($view_log) {
        $prompt
    }

    if ($LASTEXITCODE -eq 0) {
        $testsuccess = $true
    } elseif ($LASTEXITCODE -eq 1) {
        $testsuccess = "denied"
    } elseif ($LASTEXITCODE -eq 2) {
        $testsuccess = $false
    } elseif ($LASTEXITCODE -eq 5) {
        $testsuccess = "timeout"
    } else {
        $testsuccess = "unknown"
    }

    return $testsuccess
}

$netbox_user = "netbox-API-read"
$netbox_password = get-key -username $netbox_user
$connect_netbox = connect-netboxserver -username $netbox_user -password $netbox_password

if (-not $connect_netbox) {
    exit 1
}

Write-Host "Getting VM's from Netbox"
$nb_vms = get-netboxVM | Sort-Object { $_.name }
disconnect-netboxserver

$failed_scans = @()

foreach ($vm in $nb_vms) {
    if ($vm.status.value -ne "active") {
        continue
    } elseif ($vm.role.slug -in ("admin", "vmware", "fortinet")) {
        continue
    }

    Write-Host "Scanning $($vm.name)"

    $ip = $vm.primary_ip.address -replace "/.*"

    $scan_result = run_rkhunter -ssh_ip $ip

    if (-not $scan_result) {
        $failed_scans += $ip
    }
}

if ($failed_scans.count -gt 0) {
    Write-Host -ForegroundColor Red "`nSome scans failed, check /var/log/rkhunter.log at these hosts:"
    foreach ($ip in $failed_scans) {
        Write-Host -ForegroundColor Red "$ip"
    }
    exit 1
} else {
    Write-Host -ForegroundColor Green "All scans successful"
}

Expect script:

#!/usr/bin/expect -f

set SSH_IP [lindex $argv 0]

set timeout 90

expect_before {
    timeout { puts "timeout"; exit 5 }
    eof     { puts "eof";     exit 0 }
}

spawn ssh admin@$SSH_IP "sudo rkhunter -c -sk -q --summary"
expect "*Permission denied*" {
    exit 1
} "*No warnings were found while checking the system*" {
    exit 0
} "*One or more warnings have been found while checking the system*" {
    exit 2
}

expect eof

Running scripts as the rundeck user from the VM does NOT present any problems and scripts behave exactly like I expect. The powershell script executes the expect script, which takes about 60 seconds to complete per VM, as expected when running the scan. enter image description here

However when I use Rundeck to run the same script I get these errors: enter image description here

Changing the way the expect scripts exits (puts stderr eof) does not change the behavior other than is console logs eof multiple times without actually running the expect script against one VM at a time and waiting for the scan to finish.

I have search for two days to try and find a solution to get the Rundeck job to behave like the terminal does, and I think I'm going insane, I can't be the only one having issues like this?

I have tried to change how Rundeck executes the powershell script without success. I have tried to change how the expect script spawns the ssh command

spawn $env(SHELL)
send -- "ssh upsales@$SSH_IP 'sudo rkhunter -c -sk -q --summary'"

This results in a different type of error. enter image description here


Solution

  • Got the job to work thanks to @GlennJackman (I was hoping you would look at this issue)

    The updated except script that works correctly:

    #!/usr/bin/expect -f
    
    set SSH_IP [lindex $argv 0]
    
    set timeout 90
    
    set force_conservative 1
    if {$force_conservative} {
        set send_slow {1 .1}
        proc send {ignore arg} {
            sleep .1
            exp_send -s -- $arg
        }
    }
    
    spawn /bin/bash
    
    expect_before {
        timeout { puts "timeout"; exit 5 }
        eof     { exit 0 }
    }
    
    send -- "ssh admin@$SSH_IP 'sudo rkhunter -c -sk -q --summary'\r"
    expect "*Permission denied*" {
        exit 1
    } "*No warnings were found while checking the system*" {
        exit 0
    } "*One or more warnings have been found while checking the system*" {
        exit 2
    }
    
    expect eof
    

    Spawning a shell also solved an issue I had with another except script that verifies SSH access, so two birds with one stone!