pythonazure-devopssubprocesspython-sphinxsphinx-apidoc

Running sphinx-apidoc in subprocess.run() in AzureDevops Pipeline (Ubuntu 20.04) gives no Errors and does not execute


Problem

As the title says, I am trying to run sphinx-apidoc from subprocess.run() on Ubuntu 20.04 in Azure DevOps Build Pipeline.

My problem is that I seem to get an error but no message and nothing is really executed?

My code is

call = ['sphinx-apidoc']

try:
    res = subprocess.run(call, text=True, check=True, stderr=subprocess.PIPE,            stdout=subprocess.PIPE)
    print(res)

    print("stdout: ", res.stdout)
    print("stderr: ", res.stderr)

except subprocess.CalledProcessError as e:
    print("CalledProcessError: " + e.output)

and my output is

CalledProcessError:

without any output.

What I tried

I can invoke sphinx-apidoc using a pipeline step task: CmdLine@2 And I can also call for example python --version using the above subprocess.run(), using

call= ['python'] 
call.append('--version')
  1. Why is it that I do not get an output from an error?
  2. Why is it not working although other commands like running python works?
  3. why can I execute the command from a pipeline step without a problem?

Update - Task Definitions

For the test command, I just use this:

- task: CmdLine@2
  inputs:
    script: |
      sphinx-apidoc *putfolder *source

for my python script that should run Subprocess.run()

Python3.9.15

- task: PythonScript@0
  inputs:
    scriptSource: 'filePath'
    scriptPath: '$(System.DefaultWorkingDirectory)/myScript.py'

p.s. I know that only calling sphinx-apidoc without arguments will lead to an error, this is just for the sake of simplicity. And it should still give me a proper error message, so I know the subprocess was run properly.


Solution

  • Ok, so after trying a lots of different things, I want to answer myself:

    There was a magnitude of problems.

    1. sphinx-apidoc module path

    I provided an absolute module path that sphinx-apidoc does not like. Need to pass in a relative path

    2. properly use capture_output-option

    In the end, I removed the stdout=subprocess.PIPE and just set capture_output. I also removed shell=True that I had used temporarily

    3. Check command string

    I used subprocess.list2cmdline(*listOfArgs*) to verify my command, I also had to remove quotes/double quotes that were not needed.

    So had nothing to do with Azure DevOps or some wrong environment setup, just me not being able to properly develop in python + handling strings and commands under Ubuntu :D But maybe this still is of help to someone

    Final Code

    (not perfect, also shows you the command that was sent and gives you the output from the command)

    cmd = ['sphinx-apidoc']
    cmd.append('-f')
    ...          
    try:
        res = subprocess.run(callList, text=True, capture_output=True, check=True)
        print(res)
        print(res.stdout)
    
    except subprocess.CalledProcessError as e: 
        print("CalledProcessError: " + str(e.output))