pythonpipelineluigi

How to get the filename of a required previous task output in Luigi


In the Luigi's samples I have read, when you want to use the output file of a previous required task you do something like this

@requires(TaskB)
class TaskA(luigi.Task):
    def run(self):
        with self.input().open('r') as input:
            input.read()...something else etc

so you are opening the output file from a previous required taskB automatically when you call input

However I want to do something different. I need the complete file name of that file. I don't need to open it yet, just the file name to pass it as an argument to some function.

How can I get the file name of the output of a previous task?


Solution

  • I think in your case the self.input() is a LocalTarget.

    You can Try self.input().path to get the path.

    EDIT:

    If TaskB defines multiple outputs, for example a list, you would have to do:

    self.input()[0].path
    

    Or you can iterate through it. Having that said having multiple output is not recommended

    If TaskA defines multiple inputs, the way to access the inputs is explained here