pythonpexpect

How to keep Python pexpect shell open


I am trying to use pexpect to have a main processing loop that sends input to a number of other processes in order.

Code
pexpect_test.py

if __name__ == '__main__':
    count = 3
    processes = []
    for x in range(count):
        processes.append(pexpect.spawn('/bin/bash', logfile=sys.stdout.buffer))
    print(f'Processes Created: {len(processes)}')

    inputs = ['a', 'b', 'c']
    for input in inputs:
        print(f'Processing input: {input}')
        for process in processes:
            line = f'python test.py {input}'
            print(f'Sending Line: {line}')
            process.sendline(line)
            print('reading')
            res = process.read()
            print(f'res: {res}')

test.py

if __name__ == '__main__':
    print(f'Process Started: {argv[1]} ')
    sleep(2)
    print(f'Process Complete: {argv[1]}')

Results

$ python pexpect_test.py 
Processes Created: 3
Processing input: a
Sending Line: python x.py a
python x.py a
reading
$ python x.py a
Process Started: a 
Process Complete: a
$ ^CTraceback (most recent call last):

It looks like the process ends up just waiting at the spawned shell so I either have to use control+c or let it timeout. How could I leave the shells running while returning the flow of logic back to the main script?

Simplified Example

if __name__ == '__main__':
    ps = pexpect.spawn('/bin/bash', logfile=sys.stdout.buffer)
    ps.sendline('python x.py 1 a')

    # Expect EOF
    ps.expect(pexpect.EOF)
    # - The child bash shell stays active and control does not go back to python code
    # - Will eventully timeout as expected and raise an exception
    # - Does correctly call python script and waits for the defined amount of time

    # Expect terminal character
    ps.expect_exact('$ ')
    # - The wait is skipped but the child shell does not remain active

    # Expect literal string printed in child process
    ps.expect_exact('Completed')
    # - Correctly spawns child thread and waits specified duration
    # - Return control back to script
    # - The requirement to add a print statement containing unique string is not desirable. 
    #  -There could be hundreds of different processes that need to be run and my group does not manage their code.

    # Expect the terminal character twice
    ps.expect('\$.*\$ ')
    # - For some reason this works exactly as I need it to. 

Notes:


Solution

  • I was able to determine what pattern to expect with the following approach:

    if __name__ == '__main__':
        ps = pexpect.spawn('/bin/bash', logfile=sys.stdout.buffer, timeout=7)
        ps.sendline('python x.py 1 a')
        try:
            ps.expect('something')
        except:
            print(ps.before)
    

    Since the string passed to expect will match nothing, this will result in all output being in the before property which for my local machine was:

    b'python x.py 1 a\r\n(base) \x1b]0;name@name-VirtualBox: ~/path/to/src\x07\x1b[01;32mname@name-VirtualBox\x1b[00m:\x1b[01;34m~/path/to/src\x1b[00m$ python x.py 1 a\r\nProcess 1 Started: a\r\nProcess 1 Completed: a\r\n(base) \x1b]0;name@name-VirtualBox: ~/path/to/src\x07\x1b[01;32mname@name-VirtualBox\x1b[00m:\x1b[01;34m~/path/to/src\x1b[00m$ '
    

    This was different on the actual platform I need to use this for but the process will allow you to determine what you need to expect. For my local testing it was ps.expect('\$.*\$ ') but for the actual platform it was ps.expect('a-unique-string.*# ')